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ABSTRACT 

This report is divided into two parts. The first part 
contains the major sections of the author’s doctoral dissertation 
comparing the effects of two instructional problem-solving programs. 
The fifth grade students in six classes (three schools) were randomly 
assigned to the two programs: The Productive Thinking Program, a 
commercially- available sequence which develops general 
problem-solving skills and contains no mathematics; and the Modified 
Wanted-Given Program, an experimental sequence which emphasizes the 
structure of arithmetical problems. Both sequences were presented in 
programmed form and took 16 consecutive school days. Fifth grade 
students in t%io classes in a fourth school acted as a control group. 
Every student received a pretest, posttest and a follow-up test seven 
weeks later. Each test battery measured several other skills besides 
problem solving, on an analysis of covar5.ance, no significant 
differences were found between the two methods of instruction and the 
control, nor was any significant sex difference found. The second 
part bf this report reviews the variables used in previous studies of 
problem solving using teletype terminals, and then applies the same 
regression techniques to verbal problems selected from the 
dissertation study described in the first part. fMM) 
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The problem of hov; children learn to solve verbal problems in 
arithmetic has been a subject of research by the Institute and the author 
for the past several years. All the research on problem solving at the 
Institute bo date has been conducted in the context of computer-assisted 
instruction (CAl) . This report discusses problem solving using tradi- 
tional paper-aiid-pencil methods, which was part of the author's doctoral 
dissertation, and includes further regression analysis of the same data 
base using variables from previous studies in which students at teletype 
terminals served as subjects. One purpose for further analysis was to 
determine if the variables previously found to account for a large 
proportion of the variance in a CAI setting would also account for a 
large proportion of the variance in the traditional paper-and-pencil 
problem-solving setting. 

The first four chapters contain the major sections of my 
dissertation. Chapter V presents a review of the variables used in 
previous studies of problem solving with students at teletype terminals. 
Chapter VI presents the results of analysis on verbal problems selected 
from the dissertation study, using the regression techniques developed 
in earlier work. A comparison of the goodness of fit of the model is then 
made for each method. 

I would like to acknowledge the interest and assistance of Dr. E. G. 
Begle for his direction and guidance and for making computer time available 
through the School of Mathematics Study Group to analyze the data reported 
in Chapters I- IV. Mr Ray Rees of SMS G gave much of his time processing the 
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data, while indispensable friends, namely, Mrs. Arlene Dyre, Mrs. Velma 
Hoffer, and Mrs. Grace Kanz aided in administering the various treatments 
and tests used in this study. 

I am also grateful to Professor Patrick Suppes of LMSSS for providing 
needed facilities, including the Institute's PDP-10 computer services, 
materials and for partially supporting this study with funds from National 
Science Foundation Research Grants NSFG-I 8709 and NSFGJ-433X. The data 
for Chap'*'ers V-VI were proce.ssed bj" the author. 

Finally, I appreciate the assistance, patience, and encouragement of 
my committee and especially my wife, Roberta. 
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CHAPTER I 
INTRODUCTION 

One of the most important goals of mathematics instruction is to 
develop in students the ability to solve verbal problems (Kramer, 1966, 
p. 349). For the early grades, textbook writers and mathematicians 
recommend that primavy emphasis be placed on understanding the problem 
and that secondary emphasis be placed on computing the answer (Goals, 
1963, p. 36). For the later grades, they recommend that emphasis be 
placed on establishing a set of basic rules such as those that follow, 
for use in problem solving (Duncan, Capps, Dolcioni, Quest, 1967, p. 54) 

1. Identi/fying the sets involved; 

2. Determining whether the sets are to be joined, separated, or 
compared; 

3. Writing an equation that corresponds to the set operation; 

4. Solving the equation; 

5. Interpreting the solution in terms of the sets involved. 

The hope is that by applying the techniques embodied in such rules 
the students will be able to transfer the problem-solving skills learned 
in the classroom to real-life situations in later life. In most cases, 
however, the set of rules, together with the problem set to which they 
are applied, follows chapters on specific topics in the textbooks. 

Rather than creating a true problem-solving situation, the exercises 
often represent little more than a verbal application of the compute- 
tional skills introduced in a preceding chapter . 

. ,8 ^ 
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Traditionally, instruction in problem solving either centers around 
teaching students to follow some set of rules, steps, or heuristics, or 
simply eliminating all rules: "The best way to teach children how to 

solve problems is to give them lots of problems to solve (Van Engen, 

1959, p. 74)." One popular text instructs its seventh-grade readers to 
solve problems as follows (Eicholz, O'Daffer, Brumfield, Shanks, Fleenor, 
1967, p. 131): 

In working story problems, it is helpful to be 
oble to compute rapidly and accurately. It is even 
more important to be able to decide what to do with 
the numbers given in the problem. It is impossible 
to memorize rules, that will tell you how to work 
every problem you may need to solve. You simply must 
think carefully about the information given and then . 
decide what operations to perform upon the numbers. 

The text continues with several sample solutions to problems with 

the words "The following examples show how you might reason in order to 

solve difficult problems." 

Two new approaches to teaching problem solving have been advanced 
in recent years. One was a modification of the traditional "wanted- 
given" approach (Wilson, 1964). The ether emphasized problem solving at 
a general level, that is, it was not oriented to any particular academic 
discipline (Covington, Crutchfield, and Davies, 1966). Results from 
each of the programs indicated that students who completed the respec- 
tive programs made significantly greater gains on posttest measures of 
creative thinking or problem solving than did their respective control 
groups . 

The question examined in this study was whether students who had 
received training in general problem-solving skills by using The Produc - 
tive Thinking Program Series One t General Problem Solving (Covington, 
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Crutchfield, and Davies, 1966) would achieve significantly higher scores 
on pu'sttest measures of problem solving in mathematics than would stu- 
dents who had received specific training in problem solving in mathema- 
tics using a Modified Wanted-Given Program approach. 



The purpose of this study was to investigate the differential 
effects of two instructional programs on performance of verbal mathema- 
tical problems by students at the fifth-grade level. In previous studies 
using one of the programs. The Productive Thinking Program , student per- 
formance consistently improved on tests of creative thinking and general 
problem solving. In the other program a modification of a program de- 
veloped by Wilson (1964), students were instructed to solve problems in 
mathematics using a Modified Wanted-Given approach. The central question 
was whether a general approach to problem solving, such as The Productive 
Thinking Program , would produce higher performance on criterion tasks, 
primarily mathematical problem-solving tasks, than would a program which 
used a mathematical context to teach problem solving in mathematics. 

For some students, all problems are not really problems and calling 
a set of sentences a problem is somewhat arbitrary. As Cronbach (1948) 
pointed out "a situation presents a problem only when one must give a 
response (that is, when he seeks satisfaction) and has no habitual re- 
sponse which will give satisfaction, [p. 32]" The term "problem" as used 
in this study refers to a statement in written form that requires a 
written response, this definition is in agreement with that given by a 
standard reference in the field, the Mathematics Dictibnary (James and 
James, 1968), "a question proposed foV solution; a matter for examina- 
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tion; a proposition requiring an operation to be performed or a construc- 
tion to be made, as to bisect an angle or find an eighth root of 2. 

[p. 286]" The distinction between an exercise and a problem noted by 
Henderson and Pingry (1953) was observed in the course of the study. 

Their definition is as follows: 



With the exception of the syntactical form, the 
chief difference between exercises and verbal 'problems' 
lies in their intended use. Exercises, such as those 
dealing with the fundamental operations, exponents, 
radicals, the binomial theorem, and derivatives, are 
for the purpose of. teaching certain mathematical concepts 
and generalizations . Verbal problems are for the* purpose 
of teaching the generalizations relative to the process 
or method of problem solving. These have no necessary 
relation to a particular kind of mathematics problem; the 
problem-solving process is essentially the same for all 
problems [p. 235J . 

Throughout the study the emphasis in the problem-solving programs 

was on the process rather than the teaching of certain specific mathema- 

/ 

tical concepts. The terms "word problem," "stated problem;" "story pro- 
blem," or simply "problem"^ were all used synonymously. 



REVIEW OF RESEARCH ON PROBLEM SOLVING 



Research on problem solving in elementary-s|chopl mathematics has 
not been systematic and results often conflict. This brief review of 
previous studies includes only the major areas in wh^ studies have 
been conducted to giye the reader a feeling for t^^ diyersity of studies 
in problem solving in, mathematics and to show how this study is related 
to past work in the field of problem solying in ina thematic s. Initially 
the review includes studies fhht used su^^^ levels. 

Later . the reView^xbcusea^^ grades, 

; ei.g. , grades' are c^egorized, , genet^lly, . ac- 

cording to the clasHf Kilpatf ick (1969) . , 
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Problem - solving Ability 

Individual differences . Dodson (1970, p. 104) prepared a composite 
list of the strongest characteristics of a good problem solver in mathe- 
matics. Basing his evaluation on data involving approximately 1,500 
tenth- through twelfth-grade students from the National Longitudinal 
Study of Mathematical Abilities he concluded that a good problem solver: 

1. performs higher on all of the mathematics achieve- 
ment tests than the poorer problem solvers. 

2. performs high in solving mathematics problems 
: that require a great deal of synthesis. 

3. solves algebraic equations prof iciently . 

4. performs well on more advanced mathematics 
achievement tests administered a year after 
the criterion test. 

5. scores high on verbal and general. reasoning tests. . 

6. determines spatial relationships successfully . , . 

7. resists distraction, identifies .critical elements, 
and remains independent of irrelevant, elements . 



; 8. Is a divergent thinker. . >: 

9. has low dehLlltat|.ng test anxiety while his , 

10. has a positive attitude toward .mathematics,. ; 
o'Ihv 11. sees .himself ;■ as.ia.vgood'iinatheipatic^i-, student;: . 

i'i’fv,; .\jand.:: does v-not .'.wish.^^to.vbe ja.>‘be t ter/iina.thema tics ^ ' f ; -if; ,. ; 



..t -.'on ■'12v-ri'‘': has '■teachfersV;:Who-;ihad:ii;the:;^mo8t'i:credit8;;b -j -vryf!; ■ 

:;:t?the«baichelbrX8ftde 'Vsnw i 
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13. has teachers who had the highest degrees. 

14. comes from a family with a relatively high income. 

15. comes from a community in which the starting 
teacher's salary is higher than the starting salary 
of the teachers of his poorer problem-solving peers. 

16 . lives in a community that has had a recent change 
in the size of the population. 

17. has a socioeconomic index that was about the same 
as that of a poorer problem solver. 

Tate and Stanier (1964) found junior high school students differed 
in their ability to judge when they had enough facts to solve a problem. 
Poor problem solvers tended to select answers that had a high affective 
component on the practical judgment test. 

Sex differences in the problem-solving ability of ninth-grade stu- 
dents were studied by Sheehan (1968) . After five weeks of special 
treatment, small-step, ’linearly ■programmed materials and problem units 
accompanied by reference texts, significant differences in the adjusted 
scores on a problem-solving ^ task in algebra favored the boys. Sheehan 
concluded that sex did make a difference 'in -learning to: solve problems 
in algebra and other sdhdol learning as well. ' 

Few definitive studies 'on'lndividiial differences in problem-solving 
ability of students * in grades 4j 5^ and 6 have been made ; ■ 

Related sktlls ^' hhd: ;^ abilitie8 l> ~Ih a;: study using 1,400 sixth graders, 
Balow ;(1964) astiblish^ and computation 

ability correlated with problem-splytng ability as^ TO the arith- 

TCtic reasoning^ found no in- 

between computational and f con- 
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trolled. Comparing the F-ratios Balow concluded that computation is a 
much more important factor in problem solving than reading ability. A 
year earlier, however, Martin (1963) noted that the partial correlation 
between reading and problem solving with computation held constant was 
higher at both fourth- and eighth-grade levels than the partial correla- 
tion between computation and problem solving with reading held constant. 

Wederlin (1966) rotated two factorial studies to a common structure 
and identified five factors common to both studies on which the loadings 
were almost identical. He concluded that problem solving in mathematics 
depends primarily on a general reasoning factor. The deductive reason- 
ing and numerical factors were held to be of somewhat lesser importance. 
Wederlin determined that little is known concerning the nature of deduc- 
tive reasoning and numerical factors. The majority of the 371 subjects 
in both of Werdelin's studies were 13, 14, and 15 years old. Very (1967) 
found four reasoning factors, arithmetic, deductive, inductive, and gen- 
eral for males in his study involving 355 college students . By perform- 
ing separate factor analyses by sex, he affirmed that males, in-general, 
tended to have a greater number of mathematical abilities than did the 
females. Further, the abilities of the males were more specific and more 
easily identified. A later study (Dye and Very, 1968) revealed ninth- 
grade females were generally superior to males in tests of numerical 






facility and perceptual speed. At the eleventh-grade level the differ- 
ence in favor, C|f females WjBs somewhat less. At the college level males 
were superior in every testybf arithmetic, reasoning and mathematics ap- 

titude while; feiMles retained superlbrity in perceptual speed ahd fre- 

xva... -K; ■ ,.vZoZt. o' 

quently. demonstrated superiority in yerbal skills. / 

xy:- 
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Affective variables . A study involving 358 sixth graders by 
Jonsson (1965) reported some interaction between test anxiety and test 
difficulty, expecially for girls. Other results of the study suggested 
that anxiety may pxt to distort the findings in studies of problem solv- 
ing, particularly for difficult and complex problems. Difficult tests 
may have a detrimental effect on the performance of highly anxious stu- 



dents. 

In a Study conducted by Gangler (1967), college students who were 
told their work on a problem-solving task in logic would count toward 
their grade, in mathematics achieved lower scores on the task than those 
who were not told. The negative effect was greatest for high I.Q. stu- 



dents. One of the most important conclusions of this study was that 
overt participation in the learning situation apparently made the stu- 
' dents more flexible in problem solving and resulted in fewer errors. It 



was also summarized that the type of participation, overt or covert, as 
well as motivation were important in problem solving and varied accord- 
ing with the f students. , . , . V, 

Low-ability students in two, English sc^ less frustrated in 

a..;low-competitiye school enyirpnment And were also less r^id in their 
iapproach . to solving problems thaji, comparable children in a traditional 
8choplr;,(Kelln«r, . Pringle, and, McKenzie, 1965) . ; • , 






; { 



Problem-solving Tasks 

There is such variety in the types of problems used in various' stud- 
ies that generalizations" of .the ifindings of any one s'tudy' Seems unwar- 
; ranted (Kilpatrick; 1969) ^ proposed a set of 29 problems with 

ditmnsiohable independent He' Hope wbulli beconie a’ is ta ndard 

' set of mt;(teria Is to use , in studies of human ; prob lem solving . Eleven 



' ;; r; 
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years later, Davis (1966) summarized the status of research and theory 
in human problem solving by considering one category of tasks as those 
that required covert trial-and-error behavior and the other category as 
those that approach overt trial-and-error. To date, standardized sets 
of problems have not had any wide-scale use. The following categories 
include most of the types employed by contemporary studies. 

Problem content . Studies of problem structure have examined varia- 
tion in the language used in stating a word problem such as the presence 
or absence of an existential quantifier, the amount of redundancy, se- 
lectivity, and contiguity in problem statements (West and Loree, 1968), 
and placement of the question in the problem statement (Williams and 
McCrelght, 1965) . The results indicated that curriculum developers 
should insure a variety of problem statements in which they use differ- 
ent words to describe situations. Generally, problems with a high de- 
gree of selectivity (little irrelevant data) and redundancy (repeating 
the data) are easier for both seventh- and ninth-grade students. De- 
creasing the contiguity (increasing the distance between data) makes 
problems more difficult for seventh graders, but not for ninth graders. 
Asking the question first in a word-problem statement may be more help- 
ful for some fifth- and sixth-grade stude its, even though the differ- 
ences may not be significant in terms of improved performance. Problems 
are apparently easier for fifth-grade students to solve when the numeri- 
cal data are presented in order of use (Burns and Yohally,vl964) rather 
than in mixed order. Students with low arithmetic reasonihg ability 
find problems Iti nixed order more difficult . it was suggested that stu- 
dents be given problems with proper order in the initiar s of in- 

struction. The results of a study by Stull . 
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feet of auditory reading assistance suggested that problem difficulty is 
more a function of reasoning ability than reading skill for groups of 
fourth-, fifth-, and sixth-grade students . The readability of problems 
was found to interact significantly with mental ability in a study by 
Thompson (1967). The data suggested that mental ability and readability 
are each significant in their effect on problem solving. The effect of 
readability was greatest for those of low mental ability. 



Problem - solving Processes 

Deve lopmenta 1 changes . The work of Piaget has generated many stud- 
ies in recent years. Several of the studies demonstrated that young 
children can learn and transfer quite complicated problem-solving stra- 
tegies (Stern, 1967; Stern and Keislar, 1967; Wittrock, 1967). 

Studies using problems in elementary sentential logic with first, 
second, and third graders (O'Brien and Shapiro, 1968; Hill, 1960) have 
demonstrated that there are differences in the ability of young children 



to recognize logically necessary conclusions and their ability to test 
the logically necessary conclusion. While no evidence of growth in 
ability to test logical necessity was apparent, there was evidence of 
corresponding growth in ability to recognize logical necessity. 



Strategies of inquiry . To date, nearly all studies in this area 
have been concerned with concept attainment and have used non-mathema- 






tical tasks . Neimark and Lewis (1967) suggested that the acquisition 
of inforiMtion-gathering, str is essentially all-or-nohe for each 

individual. The evidence which indicates an increase in the use of in- 

formation gathering by older students may be, interpreted as simply 

meaning that more individuals acquire strategies with age. 
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Heuristic methods . Polya (1957) generated a great interest in 
heuristic methods with his set of rules for solving problems. Several 
studies on problem solving have attempted either to teach heuristic 
methods or to analyze problem-solving protocols using a system based 
on Polya's system or various other tecH-iques. Wilson (1967) stated, 
in a study involving instruction of algebraic concepts to high school 
students, that instruction should begin with presentation of the con- 
tent, be followed by task-specific heuristics and then be concluded 
with general problem-solving heuristics. Kilpatrick (1969) noted that 
in many cases the important factor is how the problem is seen by the 
student. He observed that the important question of how a student 
adapts various heuristics to different kinds of problems is yet to be 
studied in any depth. Zweng (1968, p. 253) went a step further when . 
she said "We need to know more about how children do, in fact 'see* 
the physical world." 

Creative Thinking 

Creativity and problem solving have been the subjects of a number 
o£ studies, but few have dealt directly with problem solving in mathe- 
nr:tic8. Tests developed to measure creativity or to identify creative 
students sometimes proved disappointing . (Prouse, 1967), while others 
looked promising (Evans, 1965), even though more data on their validity 
and reliability are needed before they can be used on a broad scale . 

The results of a study by Klein and Kellner (1967) Involving 130 
male, undergraduate studenta on a two-choice probability- learning task 
seemed to Indicate that highly creative students demonstrate a greater 
tendency to form hypotheses about patterns: of reinforcements than stu- 
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dents who are not as creative. Though not working with mathematical 
tasks, Eisenstadt (1966) found creative college students faster at 

i 

solving puzzle pifoblems. The approach used by creative students was 
apparently different from the approach used by noncreative students. 
Earlier, Mendelsohn and Griswold (1964) reported a positive relationship 
between creativity and the use of incidental cues in solving anagram 
problems. 

A natural question to ask at this point is whether it is possible 
to train people to think creatively and to transfer the skills learned 
to solve problems in mathematics . 

Summary 

The diverse nature of the majority of studies in problem solving 
in mathematics is evident in the partial list of categories reviewed 
above. The subjects who participated in these st’.idies varied in age 
from 5 years to college age. Taken as a whole, these studies offer 
only general recommendations of how to approach the problem^ of teaching 



fifth-grade students to solve problems in mathematics.' A summary of 
some specific recommendations is given in a later section. . ; y. - . 

The work by Covington, Crutchfield and Davies (1966) j ito be des- 
cribed in the following section, represents one new approach that is 
Worthy of investigation. Their program has apparently ;, produced signi- 
ficant gains dn tests of creative thinking and problem solving in some 
cabes , bdt not in others .' Should' their program 'prove to mate a signi- 
ficant con tributidh to' ttep’fP^te®h 89 iyi*' 8 'Ability of fifth-grade 
deftta ^in^ftetheiMtite^ cdhs^ 

‘'approach,' tha theof|' 'dhd assumptidna behind 'tit y ^ nwv ap- ' 
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proach to the improvement of instruction in problem solving in the 
upper elementary school could be designed. 

A second approach to teaching problem solving in mathematics at 
the fifth-grade level is one which employs a modified wanted-given ap- 
proach based on the work of Wilson (1964) and others. This approach 
provides instruction in problem solving in a mathematical context 
rather than the more general approach used by the program developed by 
Covington and others. Each of these programs is described in detail in 
the following sections. 

The Productive Thinking Program ; Assumptions . The program was de- 
signed to strengthen the elementary-school student's ability to think 
and it is an outgrowth of research on productive thinking conducted by 
Professors Richard S. Crutchfield and Martin V. Covington and colleagues 
in the Department of Psychology and the Institute of Personality Assess- 
ment and Research at the University of California, Berkeley. 

Crutchfield (1966) stated some causes and conditions that seem to con- 
tribute to poor problem-solving behavior by most elementary-school stu- 
dents. ^ ^ 

In short, the child in the all-too-typical school situa- 
tion is being; expected to develop efficient cognitive skills, 
tuch as those in problem-solving, under conditions where he 
is offered few opportunities for actual practice of the skill, 

i where the practice' he. does get is likely to. consist of tasks 
that are too easy, too repetitive, and seem meanihgles and 
trivial to him, where he is often rewarded for low-level perr.^ .^ 
fbrmance on these tasks, where he can often just passively - 
? 2 ;T! > llsten instead of 'actively trying ^out; the 8 r 

geta incomplete: and delayed eyaluative informati^^ 

;;>-:well. ;br;6pborly;7heUi8:-dbihgxahdi^little 

Just what- he ie doing-right and what ^ It is clear that 
r:^;<•.iv^.h6^fe8jpecthbieviathietic■■;S^■l■l;ficbuld;■;^ 

/ r ■ : -it ■ Is dbubtful Vthat any. ;c 

■ V- ■■i;' devciopeid; [p.- 







To overcome the current inadequacies in teaching cognitive skills, 



Crutchfield recommended five instructional steps to remedy the situa- 



tion (Crutchfield, 1966, pp. 65-66). They are: 



1. In the sweeping reconstruction of curriculum ma 
terials currently in progress, a good deal of emphasis 
should be put on designing the materials deliberately in 
such a way as to demand the exercise of complex cognitive 
skills of problem-solving and cognitive thinking to a de- 
gree compatible with other curricular aims. . . . 



2. Once such curriculum materials are avallablej 
they should be studied as they are tried out in the schools, 
with particular emphasis on a detailed observational analy- 
sis and evaluation of children's performances in the vari- 
ous skills. ... 



3. Better teaching techniques for the fostering of 
cognitive skills must be developed. In particular, the 
child should be helped to identify, discriminate, and un- 
derstand the nature and function of the skills involved: in 
various kinds of cognitive tasks. ... 



4. In curriculum materials and instructional efforts 
great stress should be placed on the transfer of cognitive 
skills. The child should be brought to understand the wide 
applicability of these complex skills to other subject- ^ 
matter problems and to other fields. . . . 



5., Finally, attention must be given to the develop- 
ment and' refinement in the child of an indispensable, , over- ^ ^ 
mastering cognitive skill--the ekill of organizing and manag- 
ing the many specific cognitive skills and resources one pos- ■ ' 
sesses for effectively attacking a problem. It is in patt 
the possession of such va master^ s that distinguishes the 
truly productive thinker "and creator from the merely talented 
person. ' . , 

Th e Pr od uc tl ve ‘ ‘ Thi nkl hg Program , then, is aimed at promoting the 



generalized problem-sblyiiig skills tof fif th’-, and t sixth-grade /students 
"skills which are /likely to be- ': a p^ 

in - mahy’; dif f erent ' -subj ectriMt :-;(Ci^tchf ■ 
The' ihs tructibnal Mte have Ibeen publis^^^ as pro- 

gr aiwnpd' bortklbts-; US 1 / cbnii c>^ t S t^hh tS\;;^f bllow/t he /s tory - 

line activities of; two chi Id^ 



^ir^rvrrrrTJirprrrrrr^ww cmrfyjwww 



rv"gwrmn^ r W <W/»L *fl> jt W> ‘ V * twr r j' ffKP»^wap«X^OC‘ 
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problems in a detective- like fashion under the direction of their uncle, 
Jim and Lila, a brother and sister, are Intended to be models with whom 
students identify as they read the stories and answer the questions in 
the program. The materials were written in programmed- text format to 
accomplish several purposes. These purposes are (Crutchfield, 1966, 
p. 69): 

1. To demonstrate and emphasize in problem-solving 
the value, necessity, and technic^ues of identifying and 
defining a problem properly, of asking questions and tak- 
ing time for reflection rather than leaping to conclu- 
sions, of looking carefully at details and searching for 
discrepancies, of generating many ideas and not fearing to 

' come up with what may seem 'silly' ideas, of looking 
everywhere when stymied for possible clues and sources of 
ideas, etc. 

2. To give the child an opportunity to generate 
his own ideas and to become more familiar with his own 
thought jprocesses and individual cognitive style. 

3. To give the child immediate and frequent feed- 
back to his ideas in the form of good examples of fruit- 
ful questions and hypotheses, either as confirmation of 
his own ideas or as helpful guides to his thinking. 

4. To allow the child to 'participate' in the solu- 
tion of problems' \rtth a pair of curious and imaginative' 
children who serve as identification-models. 

The training materials’ have been written in a free 
adabtation- of the programed - instructibnal form in an ef- 
fort to capitalize on several crucial advantages of this 
method: (l)i individual self ^administration, permitting 
freedom frcm direct ^^ 1 ^ teacher pressures, permit- 

ting individual self-pacing and^freedom from interruption 
of one's own- thought, etc; (2) immediate feedback for each 
chi Id cbntihgent' upon Hi s own iridi vidua 1 re sponse s ; (3) 
the requiring of active involvement by the child in the 
inaterials ; :(4) ^?tdie ii>roy^ 

plbitatibh of and accombdat^^^^ to the divergent and 
idiosyhcrat:ic ? responses ' the' childV (5)- but 

agnostic tests or 'indicators' of the sequential progress 
bfi thb>/child^8i|thinkirigj4 of Ht^ 
ing -ehcpunterbd; - etc Such book-form ‘ jprbgr^^ 

.!.>'CSn>He\admifti8tered^'in:-'b^:^fl^ 
a supplemehtary part of other c las swork, requiring little 
or nio birect;; interyentibn by ! t ^sp^cial : 

training of the teac in the y use of the : 
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In summary, the materials were designed to develop and strengthen 
a student's ability In using important skills and strategies for think- 
ing and problem solving, to Improve a student's awareness of his own 
thinking processes, and to Improve his attitude toward activities that: 
involve use of the mind . 

The first main study using the program was conducted In the 
spring of 1963 (Covington and Crutchfield, 1965) with two pairs, ex- 
perimental and control, of fifth-grade classes and one pair of sixth- 
grade classes, 195' students In all. A slx-hdur pretest of creative 
thinking tasks was given to all children. Students in the experimental 
classes were given 13 one-hour lessons in the training program while 
students In the control classes were given some stories to read and 
questions to answer the last few. days of the, treatment period before 
the posttest. The posttest was^an eight-hour battery. Five months 
later a one-hour followrup test was; given toi as many of the, fifth-, 



grade students as they were able to locate. It was reported that 
"... the 98 instructed children markedly; out-performed the 97r control 

• • 5^.' '’Vi,/"'- f.:. i':'* 

children" (p . 4) on almost all of the problem-solving tests on the 

criterion battery . The mar superiority ins true ted rchildren 

...■i ;/-v [ y irjrsi) vj 

was found in masures. of. divergent; thinking and originality ., The ; In- 

atructed studentaj also ^showed significant vpositiVen changes .in; the; degree 
to which they .^valued I problem: solyihg^^ lnstructed:>,chlldren < were su- 

perlor in achievement ipnlthex followrUp tea trcompared vi^^ atu 

dents . It was : epneiuded r the; training had> ef f ects;vQn!, avconaidbrable 
variety of problem-aolAiihg tabksfahdltha^^^^^^^^^^ 
time during which, no Instrubtion ws: giye^ 
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that boys and girls made approximately equal gains on posttest measures 
following training. 

After the favorable findings of the first study the training pro- 
gram was revised and expanded to. 16 lessons . Two internal criterion 
tests were also included in the new program "to provide data on how 
rapidly the superiority of the instructed children over the control 
children develops (p. 5) A second version of the program (Passive 
Expos ur^ was prepared with a slightly reduced response requirement. A 
third treatment provided children with only a set of rules to aid their 
thinking. These rules were presented immediately prior'*to each of the 
two internal criterion tests and at the start of the posttest sequence. 
This second study was conducted with 286 fifth*' and sixth-grade students. 
In general, the results corroborated the, findings of .the 1963 study. 

The Passive Exposure group demonstrated a satisfactory degree of profi- 
ciency on the posttest and it was concluded that, the Passive Exposure 
condition was equally effefctive with the regular, programmed version of 
the treatment. The Rules-Only group placed the lowest of the. treatment 
groups on the posttest criterion measures for the fifth grade, but still 
higher than the controls. The program was found to be equally effective 
with both boys and girls . .Rather than interpret the findings as evi- 
dence that new problem-solving skills had been instilled, Crutchfield 
concluded that the training might act to "sensitize the child to use the 

skills he already possesses (p.. 5).’’ 

Perlwps the most of the assumptions about teach- 

ing creative thinking on which Covington, Crutchfie Id, and Davies based 
the latest version o^ their program is ! by 01 ton 

23 V. 



18 



(1969), who assisted with the development o£ test materials for a large- 
scale test of the program. 

First, we assumed that virtually all students, regard- 
less of intelligence or initial level of competence, demon- 
strate a level o£ thinking that falls far short of what 
they are potentially capable, and that appropriate instruc- 
tional materials could bring about a substantial increase 
in the extent to which a student utilizes his potential 
for creative thinking. 

Second, we assumed that the skills involved in crea- 
tive thinking are general ski 11s- -that is, they cut across 
curriculum boundaries. They are general cognitive abilities, 
such as the production of original ideas, the invention of a 
unifying principle which integrates'* several disparate events, 
and the use of various strategies when one is 'stuck' on a 
complex problem. 

Third, we felt that the facilitation of creative think- 
ing could be accomplished without making major changes in 
the basic cognitive capacities of the student. Instead our 
instructional efforts would seek to develop, strengthen, and 
integrate skills and attitudes which the student already 
possessed in some measure, rather than attempting to develop 
entirely new and basically different cognitive capacities. 

Finally, although we are in favor of teaching tradi- 
tional subject matter in ways that promote creativity, we 
felt that direct , training of productive thinking skills, 
in addition to imaginative teaching of curriculum mater- 
ial, would be more effective than either of these tech- 
niques alone . 

The most exensive test of the revised program to date was conducted 
at Racine, Wisconsin in 1966. This study was designed to test "the in- 
structional limits of the materials by using them as an entirely self- 
contained program, with all forms of teacher particiapatlbn purposely 
held to a minimum." (Olton et al . , 1967, p. 31). All students in 44 of 
the 47 fifth-grade classes in the Racine Unified Schopi District No. 1 
were subjects for the experiment. Classes were rank ordered along a 
facllitatlve or nonfacilitatiye^^^^^a by a school district supervisor . 

A "facllitative environment" meant a room atmosphere In which topics 
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were discussed in a "free, creative interchange between teacher and stu- 
dents (p. 3)" Pretest and posttest batteries were developed by the 
Creative Thinking Project at. Berkeley and by Dr .. E... Paul Torrance of the 
University of Minnesoca. None of the batteries included mathematical 
problems. 

Of the 21 analyses of pretest variables, only one treatment effect 
was significant at the .05 level. The treatment group^ achieved a signi- 
ficantly greater number of solutions to the Jewel prob'lem, but did not 
achieve a correspondingly greater number of ideas or produce a higher 
quality of ideas than did the control classes. On the posttest, however, 
the treatment group scored higher (P < .05 or .01) on 12 variables while 
the control scored higher (p < .05) on only one variable. The superior- 
ity of the treatment group was apparent at all three levels of I.Q. iden- 
tified for the purposes of the study. In the single case on which the 
control group scored tignif leant ly higher than the experimental group on 

'J ' 

the posttest it was found that only 15 percent of all students were able 
to solve the problem (the X-ray problem). Due to the relatively small 
number of students who were able to solve the problem, it was suggested 



that the superiority of the control group was an artifact of the problem 



Itself (p'. 21) . The treatment seemed to be more effective with students 



in nonfacilitative environments as evidenced by the reduction of initial 



differences between facilltative and nonfacilitative classes on the post- 



test measiures compared with pretest measures . The reverse was true for 



the control group. No significant Treatment by Sex Interactions indica- 



ted that this version of the program wis also equa 11^^^^ effective with both 
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boys and girls . 01 ton et al . (196;7) : suninar^ 
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A diverse set of performance indicators, each re- 
flecting a different aspect of the total problem-solv- 
ing process, showed consistent benefits as a result of 
training. These included achievement of solution to 
problems, number and quality j of ideas produced, intel- 
lectual persistence, sensitivity to discrepant or puz- 
zling facts, use of a Master Thinking Skill, and an 
understanding of the process of thinking itself [p. 31]. 

Using eighth-grade students as subjects, Ripple and Dacey (1967) 
found evidence to support the transferability of direct training in gen- 
eralized problem solving to another task, the two-string problem. The 
treatment used in their study was a modified version of The Productive 
Thinking Program , upgraded to an eighth-grade level by eliminating some 
of the repetition in the program and adjusting the vocabulary level. 

The students who received the special treatment were able to solve the 
problems given, in the posttests significantly faster than their controls 
No significant differences were found Jbetween experimental and control 
groups on measures of creative thinking. It was noted in this study 
that the instructional effects were less potent than those reported by 
Covington and Crutchfield (1965) for grades 5 and 6. The effects. of 
their treatment were somewhat less for sixth graders than for fifth 
graders. It may be that the optimum grade level for which instruction 
in problem solving using material prepared in this format is grade 5. 
Torrance (1967) referred to a phenomenon known as the fourth-grade slump 
in creative thinking. His. studies showed that after grade 5 there is a 
■ general Iwering in the. level of achievement on tests of creative think- 
ing in several cultures, including pur own. He identified several peri- 
ods corresponding .to grades K or; 1^ 5, 7, and 12 during which students 
do riot perform as highly as expected on tests of creative thinking . It 
has been suggested (Torrance j I96i7j ;:Ripple and Dacy , 1967) that The Pro - 
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ductive Thinking Program be revised for grades 6 and beyond in an 
attempt to overcome this problem. 

Olton (1969) listed tl.e areas of attitude and motivation as two of 
the most important gains produced by The Productive Thinking Program . 

He cited evidence to show that fifth-grade students not only did improve 
attitudes toward problem solving, but that the improvement prevailed on 
a follow-up test more than six months later. 

A study by- Treffinger and Ripple (1969) failed to provide evidence 
for the transfer from the instructional materials to problem solving on 
a specially prepared set of arithmetic problems for students in grades 
4 through 7. It was acknowledged, however, that the teats were diffi- 
cult and their reliability was lower than desirable. Whjen I.Q. was con- 
trolled, there were no significant differences between control and ex- 
perimental pupils' mean scores on the verbal creativity tests . Evidence 
for transfer from the instructional materials to English was cited by 
Olton and Crutchfield (1969). Essays written by students who had com- 
pleted The Productive Thinking Program were judged more thoughtful than 
those written by a control group. Also, the essays from the experimen- 
tal group were judged superior in the amount and quality of thinking. 

In the above studies, no real test has been made of transfer of the 
skills taught in The - Productive Thinking Program to problem solving in 
mathematics in which the majority of problems required more than a singl 
step* Olten et ;al^ (l967)!moted a ; lack o effects on many of 

the performance measures :for the brief problems j those which emphasized 
dlvergerit ; thinking . ; Although no ^examples , of -the arithmetic problems 
used in the Treffinger -and Ripple study r(l969) iwere pgiyenj,,; the tit ^ 

"A Puzzle Form and a Text Problems Format" indicate that brief problems 



of at most two steps were probably used since a survey of the mathema- 
tics texts used in grades 4 through 7 furnishes abundant evidence of 
consistent lack of even two-step problems. As stated earlier, most "pro 
blems" given in most mathematics texts are simply "exercises" or appli- 
cations rather than problems that require serious thought on the part of 



the student. The training provided by The Productive Thinking Program 
should be most evident in situations where the solution to the given 
problem requires two or more steps. In short, the transfer effect of 
The Productive Thinking Program to problem solving in mathematics has 
not been tested with multiple -step problems. One purpose of this study 
was to investigate the effectiveness of The Productive Thinking Program 

V: 

on multiple-step verbal problems in arithmetic.. ’' • 

An important factor in the effectiveness of The Productive Thinking 
Program is the amount of teacher-directed discussion of each lesson. 
Coviilgton and Crutchfield (1965) and Olton et al. (1967) found that stu- 
dents achieved lip to -50 percent higher scores on posttest measures in; 
classes where the teacher discussed each lesson. When teachers partici- 
pated by discussing the lessons with students, new variables entered 
that made it difficult to examine the possible effects of the treatment 



Itself . As OUbn et al. (1967) pointed out, the severest test of the 



effectiveness of 'a set of instructiphal materials iei^ daily 

: ba si 8!^ without * teacher participatipn'.: It' was under these carefully^ con- 
trolled conditibhs that the ; perfprmance of . the experimenta 1 c lasses in 
the 01 ton ^ (1967) study surpassed tha t of t he control group on 30 of the 
40 intethal and pbsttbst -rteasutesi:- ^ ithe differences 



found were significant ,11 favored the, bxperlniental group . 
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The Productive Thinking Program has demonstrated its effectiveness 
in instruction in general problem solving for the more complex types of 
problems. Follow-up studies show that students retain this superiority 
over their controls for periods of 5 months or longer. Since most ver- 
bal problems In mathematics texts can be classified as exercises rather 
than problems, and one of the goals of instruction in mathematics is to 
teach students how to solve complex problems, I believe that The Produc 




tive Thinking Program was worthy of being used as one of the treatments 
in this study to determine its transfer' effect to problem solving in 
mathematics when problems of a more complex nature than usual, for the 
grade level, were used as criterion measures. 

We now turn to the consideration of the other treatment and the pre 
mises upon which it was based. 

The Modified Wanted - Given Program . The task of how to combine the 
most promising suggestions of previous research in problem solving in 



such a way as to derive an instructional program was not at all clear- 
cut. The findings of many of the studies which reported significance, 
in one way or another ^ were in 'doubt because of their poor experimental 
design. Kilpatrick (1969, p. 179), in reviewing some 117 studies in pro 
blem solving at all grade levels, noted an increase in the complexity of 
design in studies during the last few years. He suggested that more 
clinical studies with individual students should be conducted before be- 



ginning larger Studies because; 

. . .unfortunately, the increasing complexity of design has 
been aiccompanied by ah increasing number of methqdblbgical 
biunder8,|such as the inappropriate use of analysis of co- 
variance 4 \nd the use of oubjects as experimental units when 
intact classes haye been assighed to treatTuents. More dis- 
turbing still is the inVestlgatorsV apparent ignorance that 
statistical assumptions arevb^^ 




2k 



Gorman (1968) analyzed 293 articles and dissertations on problem 
solving in terms of their experimental design as defined by Campbell 
and Stanley (1963). 0£ the studies analyzed, 178 were removed £rom 

consideration £or not actually being studies o£ problem solving in math- 
ematics, £or not using as subjects students in grades 1-6, £or not being 
published in the years from 1925 to 1965, for lacking Internal validity, 
or for not being available for examination. The remaining 115 studies 
were closely examined for internal validity. Finally, the number of 
studies which met criteria VTas 37 . The recommendations from the accepted 
studies concerning the teaching of problem solving in grades 1-6 were 
centered in the following areas. 
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a» Effect usin^ chc following mcchbds: . 

(I) Sysccciaclc InscruccioA^ 

(a) Systcnuxctc ins true cion in vhich scudcncs arc asked 
CO explain how a problcrii is to be solved and why a 
parcicular process is appropriate produces greater 
gains in problcia solving than cere presen cation qf 
cany probleras. 

(b) The dcvclopccnc of understanding is a gradual process 
chac is aided by systocacic inscruccion in che four 
fundaccnCal processes. 

(c) The dcvelopnenc of understandings of che four funds- 
ccncal processes is a vical- factor in^che ieprove- 
cent of problea solving. 

2 

< 2 ) Incensive scudy of vocabulary 

<a) Pupils who sepdied quancicacive vocabulary using che 
dlrccc study techniques (enable che child to estab- 
lish a three-way association between the vriccen syn- 
bol» che sound of Chc cemy and ac lease one of its 
Bcanlngs) achieved slgnlficancly higher on a cest of 
arlthccclc probletn solving and concepts chan pupils 
who had not devoted special accencion Co che study 
of quancicacive vocabulary. 

(b) The direct scudy of quantitative vocabulary does not 
tend Co result in an iaprovemenc in general vocabu- 
lary or in reading comprehension. 

<c) The direct study p£ quantitative vocabulary is not 
Boro effective wltb one soX than with the other. 

w The direct study of quantitative vocabulary is a 
aethod that ia moro effective; with pup ila of abova 
^ ^ with 

pupils of below avet'Age intelligence. 



' '^Axig’eia^-Pacb, Effi:cb ;u£"tj;nderi ■ob;'thc ■ 

,and'’?erinaneh;ce':o:£:iaarh'lhg'';;<^py'>U^^^ 

Univeifbity;;"195$>'l''''^ v ' 

■■ '^Lpuis Frede 

feet ■ o£-7thq\'blMcc.'Study!'b£';'(lu%' S’*®*"". ■: v v 

iea''Solyi«ig''^i'l’lty/'^of’'Fi£tb'\Crade->upi-U dla-' V 

sertatlon^ Michlian State Untvars tty/ 1962 ),^ 









26 



(3) ' Sstlnaattn^ answers^ 

Practice ti\ cdClnaMn^ answers to arlthtnc tie problems is 
oi no more value to sl^th grade pupils than Is the tradi- 
.tional practice in the tolutlon bjp such problems. 

2 

(4) Croup experience 

Despite the superiority or the work performed by groups 
as compared to individual efforts in situations involv- 
ing written problem solving;, there Is no significant in- 
provement in the ability oC subjects to solve problems 
when trained in groups as compared to subjects who have 
worked by themselves continuously when evaluated under 
circumstances in which each subject must roly upon hie 
own resources. 

3 

(5) Cuisenaire materials 



' Use of Cuisenaire narerials in an elementary mathematics 
progran'resulted in slgntitcanr.ly less achievement .in 
computation and reasoning than was evident when such ma- 
' ' terlals were not used.' 

b» Comparison' of. methods; 

(I) Cbdperatlvc versus' individual ef fort^ 

(a) Children working together in pairs do solve more 

^' problems correctly than each child could do working 
''alone. *’ 

(b) Children working together in pairs do require more 
time to sbivb' problems rthah each child would do ’ 
working 'alone;-'-* .* 



^John Wi bickc’yv VThc Value of Estimating Answers to, Arithmetic 
problems and Examples The Elementary Sehool^^Tournal . ^pCV r(Scptembct» 
•1934), 24-31. 



^Bryce Byrne Hudgins, ••The Effects of Initial Croup Experience 
Upon Subsequent Individual Ability to Solve Arithmetic Problems^* (un- 
published Doctor's dissertation, Washington Ur4iverslty,19S8); . 

^?i^qbert Albert Passy, . ••How D.o.^ in a. { 

f led ■ Elt^ntai^Hit^ 

and ^ Coopuc^cibnal'^ thir^; trade^C^^ 

dlssqtcation, NeW York Uhiyet|slcy, IW^ 



^Semuel|;rvkiuctniniv^ 

In. Problem 'SblVii^“.- 



1944). n-99. 
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(2) Drill method wi:.h ins iiihi. ccthod^, 



(a) skill in corr.pucatlor. and solyinji verbal problems, 

io the chief soal of ins tree Mon, '-he method a 
teacher employs should be determined by her own pre- 
dilections. 

Xf more generalized outcomes of instruction, par- 
ticularly '■..e ability to think mathematically, are 
significant goals, it makes a difference how pupils 
are Caught. 

(c) Pupils of relatively low ability and good achieve- 
ment learn better under a drill method. 

(d) Pupils of relatively high ability and *ow achieve- 
ment learn better under a meaning method. 

(3) Association, analysis and vocabylary^ 

The association method, or that technique by which, diffi- 
cult or incorrect problems arc associated with a model, 
produces greater gain in st'edent performance In problem 
solving than the analysis or vocabulary methods . . (The 
analysis method refers to the .stcp-by-scep approach to 
problem solv ing while the vocabulary me t hod Involved the 
completion of oathomatlcai’ problems with the correct 
, term,) ■. 






' . • ’ ■ . , ■■ ■- 3 .. 

Dependencies, i;onvcr.tld?.al- formula, and individual 



(a) 



The conven tlonal- formula roc thod . of problem solving 
(i.c., four steps; . asked, given, how, answer), pro- 
vided the least gain in ability when compared with 
the individual (absence of. any formula) or dependen- 
cies method (graphic or dlagraffinatlcal) . 



^C. Lester Anderson, 'tjuantltatlvc' Thinking as Developed Under 

Connectlonlsts and Plold Theories of LoarnlngJ' Learning Theory in _ 
School Sltuatlbhs (Uhlyerslty of Minnesota Studies in Education, No. 2; 
Minneapolis: University of Minnesota Press, 1949). ^ 

^C. L. Thiele, VA Comparison of Throe Instructional Methods. in 

problem Solving RoBoarch oh the Fountlatioiis of American Kducntlon, 

Off iclalRcp’ort of the' Amor lean EducatlphalRosparch^^s'sqcl^ 

(Washingt'on,-\i)V C# : ; >'i^iicricah',Educacl6nal",Rcs^^ Asspclacion, 

1939).;: : , 

■■^Pau’i'’itJ’ 'ka .il'Arltliiccic- Problem 'Solving: ■\'’A''Study:''b£_';l^^^ “ ^ 

Effbctlveaess ‘of ‘ 

tor's dissarcacioA 



of ''Thtee ' Me thbdii‘ 'Pf Icm 

ition.V>teicK«'A '^olleio^^'^&iltkbii'^ynlvers 
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<b) When fourth and ecvchch graders arc considered as a 
whole, there is no difference between the results of 
Che dupcndcncics and individual BCChod&< 

<c) When the work of fourth graders alone is analysed, 
the data indicate chat the dependencies method is 
superior to the individual approach. 

(5) Formal analysis and graphic analysis^ • 

Kcichcr Che conventional (formal analysis) nor the de- 
pendencies method (graph^ analysis) produced changes 
that were staclstlcally significant with respect to che 
following: 

(a) chd' grade level oh which they were used; 

(b) Che ability levels within the grade (average, 
superior, inferior); 

(c) retention of ability ' in problem solving. ' 

2 

(6) Action Sequence^ Wantcd-Clven, and Practice-Only 

(Action Sdauence refers to a program focusing oh what 
is gO{ing.on, what .events went on; what is being dohe, 
vbac Ic done, what was the sequence of events, etc. , in 
eicuacibns' from which the meaning or "at tributes'' 'Of an 
operation is to be abstracted i this program emphasizes 
what one docs mentally or physically when one is odding, 
subtracting, etc. Hence, the operations are conceived 
of in teihas of their characteristic action-sequence. 

In other words, the operations arc relationships, or . 
hayb struc tures, chei relational attributes of which are 
accibn-scqucnccs. ' 

' Wahted-Ctven refers to the program which; focuses' bn the 
goals and ''tobls,'' the ''why'' and' ''with what," the ends 
and means, the wanted and'givens in situations from which 
tho meaning or. "attributes" of an operation is to be ab- 
stracted. This program emphasizes "why" and. "with what" 
one adds, subtracts, etc. ;Kence^ the operations arc 
conceived of in tenas of their characteristic ends-means. 
Or, in other words, the operations are relationships, or 



^Qalph D. Horsman, "A Comparison of Methods of Teaching Verbal 
Problems In Arithmetic in Grades Five, Six, Seven, and Eight" (unpub- 
lished Doctor's di66erca.cioa. University of Pittsburgh, 1940). 

^John Varner Wilson, '*The|, Rolo of St^ruc cure in Verbal Problem- , 
Solving in Arithmetic: ; Ah Analyticat ahd Expeifimenca 
Three Problem-Solylhg Prograns'l (unpubl^hed 
Syracuse Uaiversi^, i964y. ' , ''' "■ - 











have structures, the relational attributes of which ars 
vanted'givens. 

Practice-Only is a non-spec Ified structures program 
which provides no direct instruction concerning the 
problem situation.) 

The major difference In the three programs is In 
the contents of the thought process of the child as ha 
analyzes a problem and chooses the operation. 

(a) Emphasizing those attributes of the arithmetic 
operations termed the Vanted-Civen produces statis- 
tically significant improvement in yitrbal problem 
solving ability in arithmetic. 

(b) Emphasizing the Wanted-Civen attribtTtes.oJ the 
operations produces statistically significant im- 
provement In verbal problsa solving ability than 
comparable emphasis of ths Action-Sequence attrib- 
utes of the operations. 

(c> Emphasla on the Vanted-Given attributes produces a 
greater statistical Improvement in verbal problem 
solving than the mere provision of practice. 

(d) Emphasizing the Action-Sequence attributes of the 
' operations produces no statistically significant 
Improvamant In verbal problem solving. 



We will give, more consideration to Wilson's findings on following 
pages. First, however, the summaries of the Journal-published arltcles 
on problem solving In arithmetic, grades 1-8, for the years 1900-1968 by 
Suydam and Rledesel (1969) are presented. In all, Suydam, Rledesel and 
staff examined all the articles on arlttunet'lc published In 47 American 
and English journals and synthesized the findings of 1,104 of the stu- 
dies which were reports of actual experiments. Dissertations were also 
listed, but not synthesized. The findings from research carried out In 
the period 1950-1968 were emphasized, but significant findings from be- 
fore the advent of modern mathematics, 1950, were also Included. A par- 
tial list of their "Answers from Research; Problem Solving" follows. 
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How do pupils think In problem solving ? 

Studies by Stevenson (1925) and Corle^ (1958) reveal 
thst pupils often give little sttentlon to the sctual pro- 
blems; instead, they slmost rsndomly inanipulste numbers. 

The use of techniques such ss 'problems without number* 
csn often prevr t such rsndom attempts. 

Whet is the importance of the problem setting? 

Resesrchers such ss Bowman (1929, 1932), Brownell^ 

(1931), Hensell (1956), Evans (1940), Sutherland (1941), 

Wheat (1929), and Lyda and Church^ (1964) have explored 
the problem setting. Findings are mixed, with some re- 
searchers suggesting true-to-life settings while others 
suggest more imaginative settings. While the evidence 
appears to be unclear, one thing does emerge: problems 
of interest to pupils promote greater achievement in 
problem solving. With today's rapidly changing world it 
seems unreasonable that verbal problems used in 'elemen- 
tary school mathematics could sample all of the situa- 
tions that will be important to pupils now and in adult 
life. Perhaps the best suggestion for developing pro- 
blem settings is to take situations that are relevant 
for the child. Thus, a problem on space travel may be 
more 'real' to a sixth grader than a problem based upon 
the school lunch program. 

How does the order of the presentation of the process and 

numerical data affect the difficulty of multi - step problems? 

Burns and Yonally^ (1964) found that pupils made 
significantly higher scores on the test portions in which 
the numerical data were in proper solution order. Berglund- 
6ray and Young^ (1932) found that when the direction opera- 
tion^ (addition and multiplication) were used first in 
multi-step problems, the problems were easier than when 
inverse operations (subtraction and division) were used 
first. Thus, an 'addrthen-subtract * problem was easier 
than a 'subtract- then-add' problem. 

What ^ the effect of vocabulary and reading on problem solving ? 

Direct teaching of reading skills and vocabulary directly 
related to problem solving improves achievement (Robertson, 

1931; Dresker,^ 1934; Johnson, 1944;^ Treacy, 1944; 

VanderLinde , ^ 1964) . 

How docs wording affect problem difficulty ? 

Williams and McCreight^ (1965) report that pupils 
achieve slightly better when the question is asked first 
in a problem. Thus, since the majority of textbook 



study rejected by Gorman. 
^A study accepted by Gorman. 
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series place the question last, it is suggested that the 
teacher develop and use some word problems, in which the 
question is presented first. 

What is the readability of verbal problems in textbooks and jji 
experimental materials? 

Heddens and Smith (1964) and Smith and Reddens (1964) 
found that experimental materials were at a higher reading 
difficulty level than commercial textbook materials. How- 
ever, they were both at a higher level of reading diffi- 
culty than that prescribed by reading formula analysis. 

What is the place ^ understanding and problem solving ? 

Pace^ (1961) found that groups having systematic 
discussion concerning the meaning of problems made signi- 
ficant gains, Irish^ (1964) re’pdrts that children's pro- 
blem solving ability can be improved by (1) developing 
ability to generalize the meanings of the number opera- 
tions and the .relationships among these operations, and 
(2) developing an ability to formulate original statements 
to express these generalizations as they are attained. 

Should the a nswers to verba 1 problems be labeled ? 

While Ullrich (1955) found that teachers prefer label- 
ing there are many cases in which labeling may be incorrect 
mathematically. For example: 



Incorrect 

10 apples 
+ 6 apples 
16 apples 



Correct 

10 

+6 

16 apples 



Does cooperative group problem solving produce better 
achievement than individual problem solving ? 

Klugman^ "71944) found that two children working to- 
gether solved more problems correctly than pupils work- 
ing Individually. However, they took a greater deal of 
time to accomplish the problem solutions. Hudgins^ (1960) 
reported that group solutions are no better than the inde- 
pendent Solutions made by the most able member of the 
groups . 

What is the role of formal analysis jji improving problem 
solving? 

The use of some step-by-step procedures for analyzing 
problema has had wide appeal in the teaching of elementary 
school mathematics. Evidence by Stevens (1932), Mitchell^ 
(1932), Hanna .(1930), Bruch (1953), and Chase (1961) indi- 
cated that informal procedures are superior to following 
rigid steps such as the following: 'Answer each of these 
questions: (1) What is given? (2) What ia to be found? 

(3) What la to be done? (4) What is a close estimate of 
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the answer? and (5) VJhat Is the answer to the problem? ' If 
this analysis method is used, it is reconniended that only 
one or two of the steps be tried with any one problem. 

Whet techniques are helpful ^ improving Pupils* problem 

solving ability ? l . 

Studies by Wilson^ 11922), Stevenson (1924), 

Washburne (1926), Thiele^ (1939), Luchins^ (1942), Bemis 
and Trow (1942), Hall^ (1942), Klausroeier (1964), and 
Rledessel^ (1964) suggest that a number of specific tech- 
niques will aid In improving pupils' problem-solving abil- 
ity. These techniques include. (1) using drawings and 
diagrams, (2) following and discussing a model problem, 

(3) having pupils write their own problems and solve each 
others' problema, (4) using problems without numbers, 

(5) using orally presented problems, (6) emphasizing vo- 
cabulary, (7) writing mathematical aentencea, (8) using 
problems of proper difficulty level, (9) helping pupils to 
correct problems, (10) praising pupil progress, and (11) 
sequencing problem sets from easy to hard. 

A later suisroary of research on problem solving and recomwendations 
for teaching problem solving was reported by Rledesel (1969, pp. 54-55). 
A partial list of his suggestions follows. 

1. While the improvement of computation is impor- 
tant to problem-solving ability, the improvement of com- 
putation alone haa little, if any, measurable effect upon 
reasoning and problem solving. 

2. Motivation Is essential to effective learning. 

Tc assure optimal achlcvamsnt, pupils must be Interested 
Iti the problem-solving program. It has been found that 
pupils raact wall to a varlaty of problem settings. The 
use of *rasl-ll£e settings' may be helpful. Also, it has 
been found that use of problems from old U.S. and new 
foreign textbooks increases pupil interest. 

3. Children at all levels of problem-solving ability 
are receptive to supplemental *puz*le-type' or enrichment 
problems . 

4. Tha use of formal analysis— that is requiring 
pupils to answer each of tha following questions: (1) What 
is given? (2) What is to be found? (3) What operation do 
you use? (4) What is an astlmata of tha answer? (5) What 
is tha answer to tha problem?— does not produce superior 
results in problem solving. However, the use of one of the 
steps as a foeua for a lesson does ii^rove problem solving. 

For axaopls, on a given day tha task given to the pupils might 
be this: 'Make an estimate of an anawur to the problem. You 

do not have to compute your answer.' 
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5. The following teacher practices improve problem 
solving: 

a) Provide problems of appropriate dif- 
ficulty level. 

b) Guide pupils to use a method for get- 
ting started. 

c) Aid pupils in the analysis of infor- 
mation. 

d) Give pupils encouragement to proceed, 
and praise them when they perform some 
process correctly. 

e) Aid pupils in verifying final solutions. 

f) Start pupils with easy problems which they 
most certainly can get correct with a rea- 
sonable errount of effort. 

6. The problem-solving program should be started early. 
As soon as a child begins to work with sets, he can begin to 
aolve orally presented verbal problema. Thus an important 
part of the kindergarten program ahould be verbal problem 
solving. 



7. Problems may be used at various times in a unit: 
at the beginning to introduce a topic, as a unit progresses, 
and as review and maintenance. 

8. A variety of computational types should be pre- 
sented In most problem-solving lessons. When pupils find 
that all problems for a day's lesson Involve one operation, 
the task is actually only one of computational practice. 

9. Tape recordings of the textbook problems can be 
used with pupils who have difficulty reading problems. 

10 « A best technique has not been found for problem 

solving* However, the following techniques have been found 
to increase problem-solving ability: 

a) Make use of mathematical sentences in 
solving single and multi-step probleois* 

b) Make uSe of drawings and diagrams as a 
technique to help pupils solve problems. 

c) Make use of orally presented problems. 

In all the studies and reviews of studies examined thus far, there 
is general agreement that students can be taught to solve problems. 
However, as can be seen, there in dleagreement on how it should be done 
and of which reports should be accepted as representing good research. 
Formal analysis, where five or more steps are used to solve a problem. 
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has not been shown to produce significant gains in problem solving. 
However when one or two of the steps are emphasized j significant gains 
have resulted. Wilson's study (1967) compared the wanted-given approach 
to teaching problem solving with the action-sequence approach. In the 
wanted-given approach the child was to (Wilson, 1967, p. 488); 

1. Recognize the wanted-given structure of the 
problem. 

2. Express this structure as an equation. 

3. Compute by using the operation indicated 
by the equation. 

In the action-sequence program* the ^ chi Id waa to (p. 487): 

1. 'See' or recognize the real or Imagined 
action- sequence structure of a problem. 

2. Express the action sequence in an equation. 

3. Gonq>ute, using the operation Indicated if the 
equation is direct; or if the equation is indirect, 
imagine an appropriate second action sequence, ex- 
press it as an equation, and coflq>ute' using the « 
operation indicated. 

4. Check by rewriting the equation with the ans- 
wer in the proper position. 

The findings of the wllson study were cited earlier by Gorman (1967, 
p. 94) except that In his quote from the Wilson article (Wilson, 1967, 
p. 495) Gorman did not Include the qualification "For the types of one- 
stap verbal problems used in this study and for the age and grade level 
of the children Involved--" which preceded the four main conclusions. 

The point la that Wilson's study used only one-step problems. 

In Wilson's treatmant (1967, p. 55), the wanted-given etructure of 
the problem waa seen aa a meaoe-ande relationship. Wilson assumed that 
the child "saw" the structure of the problem and expraased the relation- 
ship In aquation form (p. 57). The aquations were always to be written 
by the student In direct rather than Indirect form. That Is, 12 - 5 - n 
rather than n 4 5 ■ 12. A sample problem from each category used by 
Wllson In ilia study best Illustrates his treatment. 
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Sample 1. Problem (p. 60) 

Bob had 9 marbles . Dick gave him 3 marbles . 
How many marbles did Bob have then? 

Classification 

A problem in which the size of a total is 
wanted and the sizes of its parts are given. 

Wanted-given structure of addition 

9 + 3 = n or 3 + 9 = n 

Sample 2. Problem (p. 61) 

Bob had 9 marbles . After Dick gave him 
some more marbles, he had 12 marbles. 

How many did Dick give Bob? 

Classification 

A problem in which the size of one part 
is wanted and the sizes of the total and 
the other part are given. 

Wanted-given structure of subtraction 



12 - 9 = n 

Sample 3. Problem (p. 61) 

Bob had 12 marbles. He lost 3 of them. 

How many marbles did Bob have then? 

Classification 

A problem in which the size of one pert is 
wanted and the sizes of the total and the 
other part are given. 

Wanted-given structure of subtraction 
12 * 3 « n 

Sample 4. Froblem (p. 62) 

Bob had sobm marblai. Ha lost 3 of them. 
Than he had 9 marbles left. How many marbles 
did Bob have to begin with? 

Olissification 
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A problem In which the size o£ a total la 
wanted and the sizes of Its parts are given. 

Wanted-'glven structure of subtraction 
9+3 = nor3 + 9 = n 

Unfortunately, the proportion of problems requiring each o£ the 
four basic operations In Wilson's texts was not the same. Of the 30 
Items on each of his posttest and retention tests, 9 were addition pro- 
blems, 13 were subtraction problems, 3 were multiplication problems, and 
5 were division problems. As Indicated by the examples, the exercises 
were quite easy and few purely computational errors were made. Wilson 
(1967) concluded that selecting the proper operation was of primary Im- 
portance. "The group which became superior In choosing correct opera- 
tions also became superior in obtaining the correct answers (p. 229)." 

Marilyn Zweng (1968) criticized the action-sequence program used by 
Wilson (1967) for not being flexible enough In Its instructional approach. 
That Is, although there are several ways to solve most problems, she con- 
tended that the children In the actloh-sequence group were led to believe 
there was only one correct mathematical ax>del for each problem. Even 
though Wilson's article was necessarily a shortened version of his dlssev 
tatlon and did not Include all the details, there was not enough evidence 
to support Zweng's criticism In full. However, the extent of control re- 
quired to maintain the distinctiveness of each treatMnt In Wilson's 
study did tend to restrict the flexibility more than one might expect In 
each treatment had It been carried out separately under less rigorous 
conditions. Bach treatment: required the student to recognise a number of 
situations in which the structure was different or the sctlon sequence 
varied. In the action-sequence treatment, the child vleusllted or Imag- 
ined the action taking place In the problem end was required to write 
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the corresponding mathematical sentence in exactly the order indicated by 
the action sequence (Wilson, 1967, p. 26). This required an almost exact 
translation of the action taking place into mathematical terms as ex- 
pressed by an equation. The vanted-given treatment, on the other hand, 
required the student to recognize the structure of the problem in terms 
of a direct mathematical sentence (Wilson, p. 43). In some cases ex- 
pressing the relationship in a direct equation amounted to two steps for 

/ 

students in the action-sequence treatment. They were required to imagine 
a second step, the direct equation, if the action sequence indicated an 
indirect equation. That is, if the action sequence suggested an equa- 
tion like 10 n B 23, the student had to imagine some action sequence 
that would have a direct form auch as 23 • 10 = n. The student then had 
to perform the operation indicated in the second step to find n. 

The modification of Wilson's vanted-given treatment used in the pre- 
sent study was aimed at reducing the number of cases a student had to 
keep in mind while trying to solve a problem and at the same time make 
the solution process more meaningful. As suggested earlier by Thiel 
(1939) and Mitchell (1932), analysis appears most efficient when only 
one or two steps are required. For this reason the wanted-given approach 
used in the present study emphasized the notion of two alternatives, that 
is, two steps. First, students were instructed in the use and meaning of 
the terms "sum," "addend," "product," and "factor." Following this, they 
Were given practice in solving simple equations, such as IS ‘t* 10 ■ n. 
Msxt, students wire taught that to find the value of n in an aquation 
such as 15 ^ n ■ 25 they had to solve the aquation n ■ 25 * 15. 

In a aiadlar manner, students vara taiight that to solve for n in 
25 X n a> 20 they had to solve n ■ 20 « 5. 

. ^ 43 
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The Instructional program emphasized that there were basically Just 
two kinds of problems, sum problems and product problems. A sum problem 
la one In which two or more quantities are being combined. . Once a stu- 
dent eat.ibllshed that the problem was a sum-type problem, only two oper- 
ations were possible, addition or subtraction. If the problem involved 
direct addltlpn, such as a + b = c, or If It had one of the addenda miss- 
ing (wanted) In the form a + n = c, the solution was n = c - a. Simi- 



larly for product problems, once the student determined It was a product- 
type problem, just two operations were possible, multiplication or divi- 
sion. The easier of the two operations was multiplication, multiplying 
the factors a x b ? c, to find the product. If one factor was missing 
(wanted) and the equation was the form a x n « c, then the solution was 
n = c « a. 

Beginning with one-step problems, the program led the student to 
make choices between two alternatives at each step. Keeping the number 
of alternatives to two is in keeping with the findings of earlier studies 
(Thial, 1939; Mitchell, 1932) and facilitates both understanding and in- 
struction through emphasis on vocabulary (Vanderlind, 1964). 

Figure 1 presents the decision structure of the program. At level 
1, the student must decide whether the problem is s sum- or product-type 
problem. At level 2, he again must msks a choice of whether to supply 
s sum» an addend, a product, or a factor. 

Ihs trsnsfsr of this approach to multiple-step problems is direct. 
The decision structure is simply a nestsd set of wsnted-given. sequences 
in mhieh the rules from csss to ease do not very. This sequence is pre- 
sented graphically in figure 2. for example, a factor may itself be a 
suai. The* student must find the sum (step 1) and then use the sum as one 
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verbal problem solving indicating decision 




Figure 2 , Decision structure o£ the Modified Wanted-Given Program for multi-step verbal problem 
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of the fectors to find the product <etep 2) . It le this second step thst 
1. often forgotten by fifth graders. They ere, In the experience of the 
writer, Inclined to perfotn one step and feel that they hove solved the 
problem since they have "done something." It was assumed that teaching 
students to first recognize the overall structure of the problem would 
Improve their understanding of the problem and reduce the number of oc- 

currences of only partially completed problems. 

By way of comparison, the first example used to illustrate Wilson’s 
approach <p. 60) was almost identical in structure to that used by a stu- 
dent in the present program. Rather than being classified as a problem 
in which the size of the total was wanted, the problem was classified by 
the present program as a sum problem. What was wanted was the sum. The 
addends were given. Therefore the operation was addition, 9 + 3 = 12 or 

3 + 9 = 12. 

The second example Wilson gave (p. 61) would also be classified as 
a sum problem of the form 9 + n = 12. However, since one of the addends 
was missing, the problem had to be solved by using the equation 
n = 12 - 9. 

The third case cited by Wilson (p. 61) was also a sum problem in 
terms of the present program. The equation was n + 3 = 12. The equation 
n = 12 - 3 had to be used to find the number of marbles remaining. 

The fourth case was yet another example of a sum problem. The add- 
ends were given. The student was asked to find the sum. 

Several pages from the instructional treatment which Illustrate 
various stages of the program, are Included in Appendix A. 

In suBssary, the key differences between Wilson’s program and the ap- 
proach used in this Study are: 
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1. Each problem, as a first step, was Identified 



according to one or two basic types, sum- or 
product-type problem, rather then the wanted- 



glven, pert-whole relationship used by Wilson 
(1967, p. 43). 

2. Wanted -given was taught In terms of sum, addends, 
factor, product, rather than in terms of parts 
and whole (Wilson, p. 36-38). 

3. The number of rules to be learned and remembered 
by the student was smsller as was the number of 
different types of problems (see Figure 1). 

4. The meaning of the terms "sum,” "addends," "product," 
and "factor" were stressed throughout the program. 

These terms were not emphasised In Wilson's study. 

5. Where appropriate, students were encoursged to 
write Indirect equations for the first step as 
expressions of the problem type rather than re- 
quired direct equations In each case (Wilson, p. 

43). Direct equations were treated as the ns- 
tural solution to indirect equations. That is, 

12 - 9 ^ n is the solution to 9 4* n ^ 12. 

The Instructional program itself was presented to students In pro- 
grasamd-booklet form, one booklet each day for 16 days. The program wet 
aisdlsr to that of Tte Productive Thinkina Program In approach, In that 
students were to follow a story line ebout a boy, Bill, and a nan, 

Mr. Smith, as they encountered and solved a series of mathematical pro- 
blems. The man in the story was trying to help the boy become a good 




problen solver and encouraged the boy, in turn, to help the readers to 
becooM good problen eolvers as veil. The linear progran provided for 
innediate feedback, cuea, and atep«by«atep instruction uaing anall steps. 

SDMMRT AND RTFOIBESBS 

Researchera, auch aa Olton, Crutchfield, and Covington, have argued 
for the tranaferrability of the problen-aolving skills developed through 
ayatenatic instruction in The Productive Thinkina Progran . The results 
of several studies confim their argunents in sone cases, but not in 
others. Failure to produce significantly greater achievenent levels in 
nathenatics aa transfer froa training in The Productive Thinking Progran 
in studies to date nay have been due to the fact that the nathenatical 
tasks used in sons of these studies vers too siaiple to desMnstrate the 
full effects of the training. The lack of sen differences in achievenent 
for students using the programed nsterials My be an indication of the 
teacher independence of the treataant. 

On the other hand, there are only general suggestions fron previous 
research on how one ought to go about teaching students to solve pro* 
bleas. Any educator who wishes to prepare an instructional prograa in 
problen solving nuat glean, froa the best of what others have attempted, 
the techniques he feels noat appropriate in light of previous research 
and theory. 

The purpose of this study was to test the hypotheses that: 

1. There will be no significant differences in scores on 
poattest neasures of problen solving skills between 
students who receive training in general problen solving 
techniques as presented in Tjta Productive Thinking 



Ffogff •od studsiiti who orw t<ught owro ipociflc 
•klllr £or lolvlng problems in msthmmsties in a 
toully amthsnsties context using the Modified 
Hanted*Glven Program. 

2. Students in either o£ the trsstaxmt groups, The 
Productlvs Thlnklna Program or ths Modified Wsntsd- 
Given Program, will not aehlsva aignifieantly 
higher scores on maasures o£ problem skills than 
students in control classes mho receive no spe- 
cial instruction in problem solving. 

3, Mo signi£icsnt differences mill be apparent be- 
tween the achievement of boys and girls on tests 
of problem solving skills in the treatment groups. 

Me turn now to tha design of the experiment itself . 






CHAPTER n 
EXP^IMENTAL DESIGN 

As stated In Chapter I, the purpose of this study vas to compare 
experimentally the differential effects of three instructional programs 
on problem solving and on measures of problem-solving skills of fifth- 
grade students. 



PROCEDURE 



Hiree treatments- - The Productive thinking Program, the Modified 
Wanted-Given Program, and The Control (no treatment)— were administered 
to eight classes of fifth-grade students in the Cupertino Union School 
District near San Jose, California. Six of the classes received the 
experimental treatments, on a raadom basis, and two classes served as 
controls. The three treatments euid two sex variables constit\xted the 
3 x 2 factorial design of this study, which vas as follows: 



°1 ^ °5 

°1 *2 O5 

°2 S 



where 0^ is the pretest, Productive Thinking Program treatment, 

Xg the Modified Wanted-Given Program treatment, 0^ the posttest, and 0^ 
the follow-up test idiich was given seven weeks sd^ter the posttest. The 
schedule for each event wais as follows: 

Pretest (in all four schools) Thursday, October 22, 1970. 
Treatment (in three schools) Monday, October 26 -Tuesday, 
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I 

I U6 

Noven^er 16, 1970 (there was one school holiday in 
the treatment period). 

Poatteat (in all four schools) Wednesday, Novesaber 18, 1970. 

Follow-up test (in all four schools) Thursday or Friday, 

January 7 or 8, 1971. 

Sublecta . The 261 subjects of Ihis study were all students in 
fifth-grade classes in four public schools in the Cupertino Union School 
District in Santa Clara County, California. The distribution of sub- 

! 

jects by school, class, treatment, and sex is shown in Table 1. All 
students (a total of 287 that included 26 special students identified 
and classified by the school as either gifted or educationally handi- 
capped to avoid discrimination of any sort) took part in the study. 

The data from the tests of the special students were not included in the 
analysis of the results. 

I 

I Assignment to treatment . Both instructional treatments were pre- 

sented to students in programmed booklets. In each treatment one book- 
let was given to each student each day for 16 consecutive school days. 
Booklets were randomised using a table of random numbers (Fisher and 
Yates, 1957) and handed out daily to students in the three experimental 
schools within each class beginning with the first day of the treatment 
period following the pretest. Each student remained in the program to 
which he was assigned on the first day for the remainder of the experi- 
mental period. The control classes received no treatment. 

Administration of the treatments . As noted by Olton (1969) and 
others earlier, permitting the teacher to discuss the treatment with the 
class had a significant influence on student achievement. To control 
the teacher variable, aides were hired to administer the tests and daily 
’ O 
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TABLE I 



Distribution of Students by Class, Sex, and Treataient 





Treatment Classes 


Control 

Classes 




Sex 


A 


B 


C 


D 


E 


P 


G 


H 


Total 




P* 


WC** 


P 


WG 


P 


WG 


P 


WG 


P 


WG 


P 


WG 








Boys 


7 


7 


10 


7 


4 


5 


7 


6 


8 


7 


7 


10 


20 


22 


127 


Girls 


11 


8 


6 


12 


11 


10 


A 


5 


8 


12 


10 


9 


16 


12 


134 


Subtotal 


18 


15 


16 


19 


15 


15 


11 


11 


16 


19 


17 


19 








Total 


3 


3 


35 


30 


22 


35 


36 


36 


34 


261 



®The number of students in The Productive Thinking Program, 
bxhe number of students in the Modified Wanted-Given Program. 



treatments in each class. Of the three aides, one was the wife of a 
graduate student, one had prior experience dealing with children in a 
school situation as a substitute teacher for a short time, and the other 
had served as a teacher aide in a junior high school the previous year. 
All aides were given the pretests before the experiment began to famil- 
iarise them with the procedure. The purpose of the experiment and the 
role they were to play were carefully explained. Each aide was given a 
set of printed instructions to follow the first few days of the experi- 
ment (see Appendix S). Further, aides were instructed not to discuss 
the problems with students, although they could answer sny questions and 
act supportively and pleasantly. The aides were in charge of the class 
during that period of the day when the treatments or testa were given. 
All treatments and tests were given before noon eath day In each school. 



48 



I 




i 



} 

I 



O 

ERIC 

MMilBIffriTLiU 



The aldee distributed the instructional naterisls st the beginning 
o£ each dess period. At the close of each class period the instruc* 
tional booklets vere picked up by the aides ao that no booklets reaained 
at the scaool. The classroon teacher; did not participate in the adsdn- 
istration of the treataents and generally used the tiae to catch up on 
other work; in fact the classrooa teachera did not even see a set of the 
instructional ■ateri.ila. Their only contact with the experiaent was 
during the initial iDeeting held at each achool at vhich tiae the purpoae 
of the experiaent was explained and their cooperation waa requested. 

The daily treataents were given during the regular aathesatics per- 
iod, and the entire period was used each day. The only aatheaatlcs in- 
struction the treataent classes received during the period of the ex- 
periaent was that contained in the instructional treataents. Those stu- 
dents who were assigned to ^he Productive Thinking Frograa received no 
aatheaatlcs instruction for 16 days, because that program contained none. 
Those students who were assigned to the Modified Wanted-Glven Frograa 
received instruction eaphaslslng the solution of problesM, as described 
earlier, rather than conputational skills. The control classes followed 
the state-adopted text idiich contained no special eaphasls on problus 
solving and few problen-solving exercises. The teachers of the control 
classes agreed not to Instruct students in problem solving during the 
tiae the experlasent was in progress. 

Pevelooment of treataents . The Productive Thinktnx Program 
(Covington, Crutchfield, and Davies, 1966) ia a copyrighted, conner- 
clally available program. One hundred sets of the Instructional mater- 
ials wars purchased for use In the study. The program Itself ia des- 
cribed in detail In Chapter X. 

■'54 
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Th# Modifitd Wanted •Given Progran was developed by the author for 
this study. Its content and approach are also described in Chapter I. 
Preliadnary versions of the daily lessons were prepared and tested in 
a single fifth-grade class which was in a school in the saae genetal 
socioeconoBdc area and school district as the classes that participated 
in the experieent. In all, 10 students worked through the first 10 les- 
sons of the prograsi* Based on their perfonaance and the teacher's rec- 
OBMndations, the prograa was revised. The revised version in 16 daily 
lessons constituted the Modified Wanted-Civen Prograa treatwnt. Since 
the first 10 lessons contained the instructional sequence, piloting only 
the first 10 lessons was considered sufficient. Lessons 11-16 contained 
internal tests and extensive reviews of the sMterial introduced in the 
first 10 lessons. 

Measuring instruswnts and scoring procedures * The pretest consisted 
of four scales--Pigure Classification. Working with Buebers* Ari thee tic 
Reasoning, and Hidden Figures - -which were to be covariates with the 
pog^^est and follow-up test scales. Statistics on each are given here, 
along with a brief description of the skill it vas intended to neasure. 

A copy of the pretest Is given in Appendix C. 

The Figure Classification Test is designed to measure a student's 
to discover rules that explain things. This scale is an adap- 
Ution of the Figure Classification last developed by the Educational 
Testing Service (BTS) (1962), which itself is an adaptation of a Oniver- 
city of Sorth Carolina version of Thurstone's test of the same name. 

The BTS form of the test consisted of two parts of 14 items each with a 
time limit of 8 minutes. Many of the items were thought to be too dif- 
ficult for fifth-grade students. 
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A form of the Figure Cleeel fleet ion Test was pilot tested in four 
fifth'grade classes with a total of approximately 120 studenta. These 
four fifth'grade classes did not participate in the later study. The 
nine items which correlated most highly with the total test score were 
selected for use in the present study. 

The instruction for the Figure Classification Test, together with 
examples, are given on the following page. Ho scale statistics were 
available from BTS for this test. The tiaw limit for the revised form, 
nine items, was set at 10 minutes to decrease the importance of speed 
aa a factor. 

The second scale. Working with Wumbers. was a test used in the five* 
year national Longitudinal Study of Mathematical Abilities (RLSMA) con- 
ducted by the School Mathematics Study Group at Stanford Oniversity. A 
description of the scale, the scale statistics, and a sample test item 
follow (RLSMA Reports, Ho. 4, 1968, p. 33). 



Spring Tear 2 
Crade S 



«to9 uoaKise WITH HUHSCM (12 Itaes; 20 minutes) This 
” designed to maasJVrabi I ity to perform 

numbers eeeording to written directions. It has flue Items In 

common with X3I3* 



EXAHfLC: The sum of the odd numbers less then ^ end 

the even numbers less then 9 Is 

(A) II (9) 13 (C) 2b (0) Si it) bS 



ALPHA « 
ERR.MEAS* 



0.68 

l.b9 





SCALE STATISTICS: 

MEAN • S.66 

ST.OEV* 2. be 



SAMPLE SUE* 1332 
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PX005 



FIGURE CIASSIFICATION - 1-3 



This is a test of your ability to discover rvilea that explain things* In each 
problem oii this test there are either two or three groups, each consisting of three 
figures. You ore to look for something that is the same about the three figures in 
aiv one group and for things that make the groups different from one another. 

now look at the safaple problem below. In the first line, the figures are 
divided into Group 1 and Group 2. The squares in Group 1 are shaded a^ the squares 
in Group 2 ore not shaded. In the second line a 1 has been written under each 
figure that has a shaded square as in Group 1. A 2 has been written under each 
figm*e with on unshaded square as in Group 2. 



Group 1 



Group 2 



□ 



□ 



□ 



□ 


□ 


■ 


B 


□ 


B 


□ 






^ "1 


/ 


/ 




/ 

« _ ^ a 




/ 



Now try this more difficult sample, which has three groups: 





'To' 




cP 








$ 










</ 


o> 









ine iigures xn uroup x uuuwxou wx ww — — ^ 

Group 2 consist of curved lines only. OSie figures in Group 3 consist of straight 
.ines only. As you can see, there are other details that have nothing to do with the 
•ule. The answers are: 1, 1# 2, 1, 2, 2. 

Your score on this test will be the number of figures identified correctly minus 
i fraction of the number marked Incorrectly. Hierefore, it will not be to your ad- 
rantage to guess unless you have some idea of the group to which the figure belongs. 

You will have 10 minutes to complete this part of the test. 



DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO. 



Copyright (S> 1962 by Educational Testily Service. All rl^ts reserved. 
A University of North Carolina Adaptation of a test by L. L. Thwstone 
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ITEM STATISTICS: 






ITEM INDEX 


1 


2 


3 


P 


69 


79 


52 


AOJ. P 


69 


79 


52 


BISERIAL 


56 


52 


56 


PERCENT NT 


0 


0 


0 


PAGE NO. 


185 


185 


185 


ITEM NO. 


1 


2 


3 


ITEM INDEX 


11 


12 




P 


31 


38 




ADJ. P 


31 


38 




BISERIAL 


39 


9 




PERCENT NT 


0 


2 




PAGE NO. 


188 


188 




ITEM NO. 


11 


12 





4 


5 


6 


7 


8 


9 


10 


45 


74 


42 


51 


37 


31 


17 


45 


74 


42 


51 


37 


31 


17 


39 


47 


53 


49 


48 


37 


01 


0 


0 


0 


0 


0 


0 


0 


185 


186 


186 


186 


187 


187 


187 


4 


5 


6 


7 


8 


9 


10 



\ 



The third scale, Arithmetic Reasoning , also a NLSMA test, was 
adapted from the Necessary Arithmetic Operations Test. The description 
of the test follows (NLSMA Reports, No. 4, 1968, p. 180). 



^*217 NECESSARY ARI THMETIC OPERATIONS 1 (15 items; 5 minutes) This 

scale was patterned after the French Kit Form R-l». The 
original test has items requiring both one and two operations. 
The Items In this scale require only one operation. The scale 
IS Intended to measure ability to determine what numerical 
operations are required to solve arithmetic problems without 

‘o carry out the computations. It is similar 
to PY222, PY610, PY701, and P2222. 



EXAMPLE: Jane's father was 26 years old when she was born. 

Jane is now 8 years old. How old is her father now? 

(A) subtract (C) add 

(B) divide (0) multiply 



SCALE STATISTICS: 

MEAN > 10.80 

ST. DEV- 3.56 



ALPHA > U.84 
ERR.MEAS- 1.41 



SAMPLE SIZE- 1465 



^58 



53 



ITEM STAtlSTICS: 



ITEM INDEX 


1 


2 


3 


P 


94 


84 


84 


ADJ. P 


94 


84 


84 


BISERIAL 


43 


61 


67 


PERCENT NT 


0 


0 


0 


PAGE NO. 


117 


117 


117 


ITEM NO. 


1 


2 


3 


ITEM INDEX 


11 


12 


13 


P 


60 


54 


65 


ADJ. P 


67 


65 


82 


BISERIAL 


53 


41 


82 


PERCENT NT 


11 


16 


21 


PAGE NO. 


119 


119 


119 


ITEM NO. 


11 


12 


13 



4 


5 


e 


7 


8 


9 


10 


78 


67 


88 


64 


89 


79 


75 


79 


67 


89 


65 


92 


83 


81 


70 


53 


68 


55 


73 


73 


75 


0 


1 


1 


2 


3 


5 


7 


117 


117 


118 


118 


118 


118 


118 


4 


5 


6 


7 


8 


9 


10 



14 15 

61 37 

83 53 

61 35 

26 31 

119 119 

14 15 



There Is a possible speed factor In this scale. The interpre- 
tation of the alpha and the error of measurement Is questionable. 



The final scale on the pretest. Hidden Figures 1 , was the NLSMA 



version of a test by the same name developed by ETS. A description of 
the test follows (NLSMA Reports, No. 4, 1968, p. 181). 



*X218 HIDDEN FIGURES 1 (16 Items; 15 minutes) This , _ , _ _ , 

patterned after the French Kit Form Cf-1 . • ^he orlglna 

test has about twice as many distracting lines n each figure. 
It is designed to measure ability to keep a definite conflgu 
ration in mind so as to make identification in spite P*'’" 
ceptual distractors. The task is to decide which of five 
geometrical figures is embedded in a complex pattern. It s 
similar to PX820, PY223, PY819, and PZ223- 



EXAMPLE: 




scale STATISTICS; 



MEAN • 
ST.DEV- 



ALPHA - 0.81 
ERR.MEAS> 1.70 



'59 



7.49 

3.92 



SAMPLE SIZE- 1461 



ITEM STATISTICS: 



ITEM INDEX 


1 


2 


3 


P 


72 


52 


58 


ADJ. P 


72 


52 


58 


BISERIAL 


41 


50 


55 


PERCENT NT 


0 


0 


0 


PAGE NO. 


105 


105 


105 


ITEM NO. 


1 


2 


3 


ITEM INDEX 


11 


12 


13 


P 


46 


58 


33 


ADJ. P 


51 


66 


40 


BISERIAL 


41 


53 


56 


PERCENT NT 


11 


13 


18 


PAGE NO. 


107 


107 


108 


ITEM NO. 


11 


12 


13 



4 


5 


6 


7 


8 


9 


10 


61 


51 


36 


39 


55 


53 


60 


61 


32 


37 


40 


57 


56 


65 


51 


52 


38 


56 


58 


62 


60 


0 


1 


2 


3 


4 


5 


7 


105 


106 


106 


106 


106 


107 


107 


4 


5 


6 


7 


8 


9 


10 


14 


15 


16 










27 


32 


18 










36 


43 


27 










50 


73 


59 










24 


27 


34 










108 


108 


108 










14 


15 


16 











There Is a possib1<; speed factor In this scale. The Interpre- 
tation of the a1pK{. and the error of measurement Is questionable. 



A complete copy of the pretest is Included in Appendix C. 

The posttest consisted of two NLSMA tests, Working with Numbers and 
Five Dots and a set of five word problems developed specifically for the 
study. 

The first scale in the posttest was an expanded version of the 
Working with Numbers Test used in the pretest. While the scale as des- 
cribed below consists of eight items, only the first three items are not 
duplicated in the pretest form of the scale. The posttest scale called 
Working with Numbers is X307 plus items 1, 2, and 3 of X313. This scale 
was labeled X307P to distinguish it from the pretest scale. The three 
items were included as problems numbered 11, 12, and 13 on the posttest 



55 

scale of 15 icena. The scale statistics for X313 are given below (KLSMA 
Reports, No. 4, 1968, p. 35). 



X3I3 ANALYSIS I (8 Items) This scale Is Intended to measure 

a b i I i t y to analyze a problem situation and to apply knowledge 
appropriately. This scale has two Items In common with X30l| 
and five I terns In common with X307. It Is the same as X6l I . 



EXAMPLE: What number does O stand for If 

3 X '4 X 5 - 12 X O 's * statement 7 

(A) 20 (B) 0 (C) 3 (D) k (E) 5 



SCALE STATISTICS: 

MEAN « 3.49 ALPHA « 0.55 SAMPLE SIZE- 1332 

ST.DEV- 1.85 ERR.MEAS- 1.24 



ITEM STATISTICS: 



ITEM INDEX 


1 


2 


3 


4 


5 


6 


7 


8 


P 


34 


24 


77 


52 


42 


51 


31 


38 


ADJ. P 


34 


25 


77 


52 


42 


51 


31 


38 


BISERIAL 


17 


34 


42 


50 


47 


41 


34 


09 


PERCENT NT 


0 


3 


0 


0 


0 


0 


0 


2 


PAGE NO. 


172 


176 


181 


185 


186 


186 


188 


188 



ITEM NO. 4 12 23 3 6 7 11 12 



Th« ••cood scale in th« post teat vaa called Five Dota . The des- 
cription of the scale and statistics arc given here (NLSMA Reports, No. 
4. 1968, p. 34). 



X308 FIVE DOTS (19 Items; 15 minutes) This scale is designed to 
measure ability to read a passage about a mathematical Idaa 
which Is unfamiliar and to answer a series of questions about 
this Idea. (No sample Item has been Included since the Intro- 
ductory explanation Is so lengthy.) It Is constructed to be 
logically parallel to X7I5. 



SCALE STATISTICS: 

MEAN - 10.50 ALPHA - 0:S5 SAMPLE SIZE- 1330 

ST.DEV- 4.68 ERR.MEAS- 1.78 



ITEM STATISTICS: 



ITEM 


INDEX 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


P 




76 


61 


81 


79 


70 


66 


77 


51 


66 


61 


ADJ. 


P 


76 


61 


81 


79 


70 


66 


77 


51 


67 


61 


BISERIAL 


60 


52 


69 


59 


59 


61 


69 


68 


68 


66 


PERCENT NT 


0 


0 


0 


0 


0 


0 


0 


5 


0 


1 


PAGE 


NO. 


191 


191 


191 


191 


191 


191 


191 


19Z 


192 


192 


ITEM 


NO. 


13 


14 


15 


16 


17 


18 


19 


20 


21 


22 


ITEM 


INDEX 


11 


12 


13 


14 


15 


16 


17 


18 


19 




P 




62 


41 


33 


34 


35 


37 


34 


46 


41 




ADJ. 


P 


63 


41 


33 


36 


36 


40 


37 


52 


47 




BISERIAL 


68 


52 


54 


43 


48 


61 


49 


47 


46 




PERCENT NT 


1 


1 


2 


4 


5 


6 


9 


11 


13 




PAGE 


NO. 


192 


192 


192 


193 


193 


193 


193 


193 


193 




ITEM 


NO. 


23 


24 


25 


26 


27 


28 


29 


30 


31 





The third scale on the posttest was a set of five arithmetic word 
problems. To obtain the problems for this scale, the author pilot- 
tested 150 word problems in three different forms in two classes at the 
fourth-, fifth-, and sixth-grade levels. Thirty of the problems were 
pilot tested in one class at the fourth-, sixth-, and eighth-grade 
levels to determine the relative difficulty of the problems. None of 
the classes or schools in which pilot testing was done was included in 



57 



the nain study. Five problesis were selected for use on the posttest. 
The problesw were chosen as being representative of their level of dif- 
ficulty, in terns of the probability correct in the pilot test, and 
their conplexlty in terms of the number of steps reotjired for solution. 
A copy of the problems in this scale, together with the complete post- 
test, is included in Append* D. 

The follow-up test consisted of thrP'i - rales. The first scale. 
Letter Puzzles I was a NLSMA scale. Its description and statistics are 
given below (NLSMA Reports, No. 4, 1968, p. 53, 54). 



X60l LETTER PUZZLES 1 (22 Items; 8 minutes) This scale Is designed 

to measure ability to handle a novel mathematical situation. 

It Is composed of addition and subtraction problems in which 
some or all of the digits have been replaced by letters. The 
task Is to determine all the missing digits. There are nine 
problems with 22 answers which are not Independent. All items 
ar€ completion Items# This scale Is similar to YoOl. 



EXAMPLE: ABB 

+ CB 

i A - _______ ® ■ ___ ^ " 



SCALE STATISTICS: 

MEAN ■ 10.68 ALPHA ■ 0.90 SAMPLE SIZE- 1138 

ST. DEV- 5.71 ERR.MEAS- 1.82 





ITEM STATISTICS: 



ITEM INDEX 


1 


2 


3 


P 


81 


75 


72 


ADJ. P 


81 


75 


72 


BISERIAL 


63 


60 


52 


PERCENV NT 


0 


0 


0 


PAGE NO. 


268 


268 


268 


ITEM NO. 


1 


2 


3 



4 


5 


6 


7 


8 


9 


10 


79 


43 


33 


34 


74 


75 


49 


79 


43 


34 


35 


75 


76 


50 


53 


66 


74 


70 


68 


69 


68 


0 


1 


1 


1 


1 


1 


2 


268 


268 


268 


268 


268 


268 


268 


4 


5 


6 


7 


8 


9 


10 



(cont I nucd ) 



This scale Is composed of nine two~ and three~Item clusters 
rather than 22 separate Items. The alpha calculated with 
cluster scores rather than Item scores Is O. 83 . 



There Is a possible speed factor In this scale. The Interpre- 
tation of the alpha and the error of measurement Is questionable. 



ITEM INDEX 


11 


12 


13 


14 


15 


16 


17 


18 


19 


20 


P 


54 


C3 


48 


42 


45 


40 


31 


27 


30 


39 


ADJ. P 


55 


66 


50 


44 


51 


45 


39 


34 


38 


57 


BISERIAL 


67 


45 


64 


65 


76 


79 


76 


77 


73 


37 


PERCENT NT 


2 


5 


5 


5 


11 


11 


21 


21 


21 


32 


PAGE NO. 


268 


263 


268 


268 


269 


269 


269 


269 


269 


269 


ITEM NO. 


11 


12 


13 


14 


15 


16 


17 


18 


19 


20 


ITEM INDEX 


21 


22 



















P 


20 


15 


ADJ. P 


29 


22 


BISERIAL 


58 


53 


PERCENT NT 


32 


32 


PAGE NO. 


269 


269 


ITEM NO. 


21 


22 
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The second scelt on the follow-up test. Directions, was another 
of the NLSMA tests. Its description and statistics are as follows 
CNLSMA Reports, No. 4, 1968, p. 85). 



X715 DIRECTIONS (24 Items; 
measure abl I I ty to re* 
which Is unfamiliar an 
this Idea. (No sample 
ductory explanation Is 
logical ly pafsl lei to 



15 minutes) This scale Is designed to 
d a passage about a mathematical idea 
d to answer a series of questions about 
Item has been Included since the Intro- 
so lengthy.) It Is constructed to be 
X308. 



SCALE STATISTICS: 

MEAN “ 15.16 ALPHA ■ 0.90 SAMPLE SIZE" 1111 

ST. DEV- 6.14 ERR.MEAS- 1.93 
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The third sc^le in the follow-up teat was a parallel form of the 
five-item word-problem test given in the posttest. The problems were 
identical in every respect except that the names of the <}uantities in 
the problems were changed. The numerical values, the number of opera- 
tions to solve each problem, and the order in which the problem state- 
ment was given were parallel. Each problem and its parallel had the 
same number of words. The problem-solving scale and the complete form 
of the follow-up test are included in Appendix E. 

Ana lysis . The data were analysed using covariance analysis. The 
assumptions upon which the analysis of covariance rests are (a) students 
were assigned to methods randomly, (b) achievement scores had a linear 
regression on the covariates within each method, (c) achievement scores 
had a normal distribution for students with the same ability in the same 
treatment, (d) the slope of the regression lines was the same for each 
method, (e) variances were homogeneous, and (f) achievement scores were 
a linear combination of independent components that included an overall 
mean, the treatment effect, a linear regression on the covariate, and an 
error term (Elashoff, 1968; Cochran, 1957). 

In the present study, both groups were treated exactly the same ex- 
cept for treatment; since the booklets were handed out on a random basis 
to students in each class. The control group received no treatment. 
Concerning the assumptions stated above, there was no indication that 
the regressions used were not linear. The covariate scales were inde- 
pendent of treatment effects since they were administered before the 
treatments began. . The achievement scores were assumed normal for each 
group of students. The sample sizes were such that this assumption 
seemed valid. The homogeneity of the regression lines were automati- 
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cally checked by the computer program used to analyze the data. The 
variances were also assumed to be homogeneous, and the achievement 
scores were assumed to be a linear combination of the components listed 
above. 

The analysis of covariance was run using the CMDX64 program 
(U.C.L.A., 1967) on an IBM 360/67 computer at the Stanford Computation 
Center by Mr. Ray Rees, Data Analysis Statistician for School Mathema- 
tics Study Group. 

The covariates were the four pretest scales. The variates were the 
scale scores on the posttest and follow-up test. 



CHAPTER III 



V; 

V 

i. 

\ 

K- 

I RESULTS 

This chapter begins with consideration of the reliability of the 

I tests used, moves to the consideration of their inter-correlation and 

the tests of some assumptions underlying analysis of covariance, and 
then presents data on the treatment effects on the variate scales for 
each covariate. 

In Chapter II the pretests, posttests, and follow-up tests were 
described and the Cronbach alpha for each, where available, was given. 
Table 2 presents for each test the alpha from the NLSMA data and the 
alpha computed from the data of the present study. For the Figure Clas- 
sification Test, no alpha was available from the NLSMA data since the 
test was not a part of that study. Similiarly, the word-problem tests, 
P501 and F502, were developed specifically for this study and therefore 
were not comparable with any existing test. As can be seen from Table 2, 
the two sets of alphas are comparable although the sample size in the 
present study is considerably smaller than that in the NLSliA study. 

The correlation coefficients among all criterion tests are presen- 
ted in Table 3. All of the correlations are significant at the .01 level 
which indicates that the tests apparently measured similar performances. 
The figures in parentheses give the number of students in each group in- 
dicated by the intersection of row and column. 

The correlation coefficients of each of the criterion tests and the 
covariates (pretests) are presented in Table 4. The N in this table indi 

I 

cates the number of students for whom a complete set of data on all four 
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Values for Cronbach's a and N f jr each Test from 
the NLSMA* Data and the Present Study 







Cronbach CC 






NLSMA Study 
a N-^ 


Present Study 
a N 


Pretests 






.70 


285 


Figure Classification 


-- 


- - 


(X005) 

Working with Numbers 


.68 


1332 


.59 


285 


(X307) 

Necessary Arithmetic 


.84 


1465 


.84 


285 


Operations ( PX217 ) 

Hidden Figures I (PX2l8) 


.81 


l46l 


.77 


285 


Posttests 








265 


Working with Numbers 


. .68/. 55 


1322 


.65 


(X307P) 

Five Dots (X308) 


.85 


1330 


.82 


265 


Word ;Problems (P50l) 


-- 


— — 


.43 


265 


Follow-up Tests 






.88 


260 


Letter Puzzles I (x60l) 


.90 


1138 


Directions (X715) 


.90 


1111 


.79 


265 


Word Problems (F502) 






.49 


265 



*Natlonal Longitudinal Study of Mathematical Abilities 
•>H«Total N for the present study 
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TABLE 3 



Correlation Matrix for Posttest and Follow-up Test Scales 



Scales 




1 


2 


3 


4 


5 


6 


Working with Numbers 
(X307P) 


1 


1.000 

(244) 


0.548 

(244) 


0.426 

(241) 


0.397 

(225) 


0.435 

(227) 


0.500 

(228) 


Five Dots 
(X308) 


2 




1.000 

(244) 


0.385 

(241) 


0.475 

(225) 


0.507 

(227) 


0.434 

(228) 


Word Problems 
(P501) 


3 






1.000 

(241) 


0.240 

(222) 


0.336 

(224) 


0.523 

(225) 


Letter Puzzles 

(x6oi) 


4 








1.000 

(233) 


0.407 

(232) 


0.389 

(233) 


Directions 

(X715) 


5 










1.000 

(235) 


0.438 

(235) 


Word Problems 
(F502) 


6 












1.000 

(236) 



Note: The figures in parentheses indicate number of students in each confutation. 

P(|rl> .19) < .001 for N ^ 244 
P(|rl> .20) < .001 for N = 222 



\ .. 




!' t* ' • 
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TABLE 4 



Correlation Coefficients for Each Posttest and Follow-up Test 
Scale with the Covariate Scales for all Students 



Scales 




1 


2 


3 


4 


N 


Figure Classification (X005) 


1 












Working with Niimbers (X307) 


2 


-03 










Necessary Arithmetic Operations 


3 


11 


50 








(PX217) 














Hidden Figures (PX2l8) 


4 


-05 


49 


' 35 






Working with Numbers (X307P) 




-09 


66 


53 


47 


240 


Five Dots (X308) 




00 


48 


45 


'33 


240 


Word Problems (P50l) 




-i6 


32 


48 


26 


237 


Letter Puzzles (x60l) 




-01 


45 


42 


32 


229 


Directions (X715) 




02 


42 


40 


36 


231 


Word Problems (F502) 




-o4 


47 


58 


37 


232 



Note: Decimal points omitted. 

P(lrl > .13) < .05# > .1?) < .01; P( lrJ > .22) < .001 

in a two-tailed test. 
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covariates and the criterion tests was obtained. For Word Problems (P501), 
the N of 237 indicates that 237 students had complete sets of data for 
all four covariate scales and the Word Problem (P501) scale. The fig- 
ures indicating the significance level in Table 4 were compiled using 
the smallest value of N. Hence, for the larger N the values are at 
least at the same level of significance as those shown. Except for the 
Figure Classification Scale (X005) , the correlation coefficients were 
all significant at the .001 level. The low or negative correlation of 
the Figure Classification Scale (X005) was surprising when compared with 
the other scales, since that scale was intended to be a measure of a 
student's ability to discover rules and explain things. It had been 
assumed that this scale would be positively and strongly correlated with 
the .other scales used. The correlation coefficients in Table 4 also in- 
dicate that the Figure Classification Scale (X005) was not positively 
and strongly correlated with other pretest scales. Apparently the fac- 
tors measured by each of the other pretest scales were quite different 
from those measured on the Figure Classification Scale (X005) . 

The correlation coefficients for each test for each treatment with 
each of the covariates are given in Table 5. The figure for N in each 
treatment group indicates the number of students in each group for whom 
there were complete sets of data for the four covariates and the criter- 
ion test shown. Several patterns are evident. First is the negative cor- 
relation of the Figure Classification Scale (X005) when compared with the 
other tests for students in both the Control and Wanted-Given Treatments. 
This strong negative relationship is not evident for students in The Pro - 
ductive Thinking Program . For boys and girls, the covariate with the 




greatest number of highly significant correlations Is Necessary Arithmetic 

> 72 



6 ? 



1 




Operations followed by Working with Numbers. The number of significant 
corrt ations (P < .01) of variate scales with covariate scales was 
highest for girls in the Modified Wantecl-Given P;i:ogram. The only covari- 
ate scale which was consistently highly correlated (P < .01) with Word 
Problems (P501) was Necessary Arithmetic Operations. With the exception 
of boys in the control group a significant correlation (P < .01) exists 
between the parallel form of the word problem scale (F502) and the same 
covariate, Necessary Arithmetic Operations. Differences in the number 
and pattern of correlation between criterion tests and covariates for the 
different treatment groups are also evident when examining Table 5, 
Overall, the greatest number of significant correlations (P < .01) was 
found in the data for students who received the Modified Wanted-Given 
Program. 

The F-ratios in each row of the first four columns of Table 6 were 
computed using a one-way analysis of variance. The small differences in 
F-values in the first four columns were duo to the variation in sample 
size in each cell as described earlier. Significant relationships were 
found to exist between the variate scales and two of the covariate scales. 
Figure Classification and Necessary Arithmetic Operations. The finding 
that significant differences did exist between covariate scales suggests 
that the use of covariance techniques for data analysis was appropriate. 

The F-values for regression in Table 6 were computed using analysis 
of covariance. As may Le seen in the Regression Colunn there were no 
significant differences due to regression. This indicates a lack of 
heterogeneity and thereby satisfies one of the basic assumptions of the 
covariance model. The complete analysis of variance tables for the co- 
variance analysis are included in Appendix F. 



TABLE 6 
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F-Ratios by Posttest and Follow-up Test Scale 
for Each Co-'rariate and the Regression 



Cover i at es 



Scale 


Fig. 

Class. 


Work. 

with 

Nos. 


Nec. 

Arith. 

Oper. 


Hidden 

Figures 

I 


Regression 


Posttest 


Working with Numbers 
(X30T7P) 


3.02* 


1.49 


3.19*»^ 


0.30 


0.706 


Five Dot (X308) 


3.02* 


1.49 


3 ^ 19 ** 


0.30 


1.559 


Word Problems (P50l) 


2.98* 


1.46 


3.06* 


0.31 


0.869 


Follow-up Test 


Letter Puzzles I 

(x6oi) 


4.17** 


0.91 


2.78* 


0.20 


1.083 


Directions (X715) 


4.26*** 


1.14 


2.98* 


0.22 


0.802 


Word Problems (F502) 


4^ 30*** 


1.07 


2.95* 


0.22 


1.423 



*P < .05 
^<*P < .01 
*^:-*P < .001 
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The raw score means and standard deviations of criterion scales and 
covarlates are shov/n by criterion test and treatment group criterion In 
Table 7. The adjusted score means for each treatment group are presented 
In Table 8. The effects of the covariance adjustment on the means ^nay be 
seen by comparing corresponding entries from Tables 7 and 8. 

The F-ratlos for scale uiean, treatment effect, sex, treatment x 
sex Interaction, and covarlates with each criterion test are showt.' In 
Table 9. One of the mean scores, the mean for Word Problems (F502), was 
not significantly different from zero. This was a surprising result. 

There were no significant differences due to treatment on any of the 
variate scales at the .05 level or beyond. Significance at the 0.1 
level on the follow-up test. Word Problems (F502), wrs Indicated for 
students In the Modified Wanted-Glven Program. The single significant 
treatment x sex Interaction (P < .05) was found for the posttest. Word 
Problems (P501). Boys In the Modified Wanted-Glven Program gained sig- 
nificantly more than did the girls receiving that treatment. 

The combination of covarlates, labeled Total Covarlates In Table 9, 
had a highly significant (P < .005) relation to the criterion scores. 

When co islderad separately., Covarlate 3, Nc::essary Arithmetic Operations, 
was also found to have a significant (P < .005) relationship to each cri- 
terion score. None of the other covarlates was significantly related to 
all tho. criterion secres. However, all covarlates were significantly re- 
lated (at least P < .01) to the posttest. Working with Numbers (X307P) . 

Tne F-statlstlc for Wo?td Problems (P501) and Covarlate 1, Figure Classifi- 
cation, was significant (P < .01) for the posttest, but not for the follow 
up test (F502) . The scale Workln^^ with Numbers (X307) was not signifi- 
cantly related to Word Vroblems (P501) on the posttest, but was slgnlfl- 
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TABLE 8 



Adjusted Means for Each Posttest and Follow-up Scale 
by Treatment Group 



1 

i Scale 


Productive 

Thinking 

Program 


Modified" 
Wanted- Given 
Program 


Control 


Total 


i 


Boy 


Girl 


Boy 


Girl 


Boy 


Girl 




1 Posttest 
















( 

i Working with Numbers 

1 (X 307 P) 


6.30 


6.74 


6.74 


6.32 


5.95 


6.13 


6.40 


i Five Dots (X 308 ) 


11.42 


10.80 


11.81 


10.83 


11.60 


10.17 


11 . ar 


1 Word Problems (P 50 l) 

1 


1.17 


1.63 


1.35 


1.50 


1.42 


1.51 


1.52 


f 

1 Follow-up Test 
















Letter Puzzles (x60l) 


10.74 


10.24 


10.48 


9.59 


10.37 


10.06 


10.34 


t • 

1 Directions (X 715 ) 


13.68 


12.90 


14.02 


13.52 


13.92 


12.71 


13.44 


Word Problems (F 502 ) 


1.53 


1.64 


1.99 


1.79 


1.52 


1.54 


1.69 



1 

I 
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cantly related (P < .005) to the parallel form (F502) in the follow-up 
test. This would seem to indicate an increase in the influence of com- 
putational skills on problem solving over time. The .covariate scale 
Hidden Figures had a marginal statistical relationship (P < .10) with the 
follow-up Word Problem test (F502). 

Mean gain scores for two sets of scales were tested for signifi- 
cance . First, an item analysis was run on Working with Numbers (X307P) 
after the three items which distinguished it from X307 were removed. 

The two tests were then identical. One ‘test was given as one of the 
pretests. Four weeks later the other was given as a posttest. A t- 
test of the mean gain scores on the two tests was computed. No signifi- 
cant differences in mean gain scores were found for any of the three 
treatment groups on X307 . 

Mean gains for the Word Problem test (P501 and F 502) were also 
tested for significance. Each of these tests consisted of five word 
problems. The only differences between items in each scale were in the 
names of things used in each problem. The computation required, 'the num- 
bers given, the order of problems, and the number of words in each pair 
of problems were identical. There were no significant mean gains in 
terms of mean number correct in Word Problem scores over the seven-week 
period between the posttest and follow-up test. The data for the t-tests 
are presented in Table 10* 

The lack of significant differences due to treatment .is consistent 
with the null hypothesizedi wlieh one; considers the= findings from the 
standpoint of mean number ^ correct on; criterion tests* The Word-Problem 
tests were more difficult, on the average, than anticipated. The re- 
sults from the analysis of mean number of problems correct were in- 
conclusive, but favored| t||e Modified Wanted-Given approach. 
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TABLE 10 



Mean Gain Scores on Two Scales; 
Working with Numbers and Word Problems 



Scale 


Treatment 

Group 


N 


Pretest 


Posttest 


Posttest- 

Pretest 


t 


df. 




1 


&r 


4.48 


4.78 


0.30 


1.28 


86 


Working with Nbnibers 


2 


93 


4.59 


4.81 


0.22 


1.04 


92 


X307P-X307 


3 


64 


5.36 


5.11 


- 0.25 


-0.12 


64 


Total 




244 


4.75 


4.88 


0.13 


0.97 


243 


Word Problems 


1 


77 


1.36 


1.51 


0.15 


1.00 


76 


F502-P501 


2 


89 


1.73 


1.83 


0.10 


0.75 


88 




3 


59 


1.61 


1.75 


0.l4 


0.88 


58 


Total 




225 


1.57 


1.70 


0.12 


1.51 


224 
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To get some Insight Into whether students tried to follow proce- 
dures presented in the instructional treatments in solving each problem, 
regardless of whether their computations were right or wrong, all sets 
of word-problem tests (P501 and F502) were analyzed. A correct procedure 
was defined as a series of mathematical operations that would lead to a 
correct solution if the computation was correct at each step. Using the 
correct procedure was independent of whether or not mathematical senten* 
ces were used to "set up" the problem. 

A new analysis of covariance was run to examine the effect of treat- 
ment on procedure for all students on the posttest scale Word Problems 
(P501). A significant difference was found (P < .001, F 2 ^i 99 = 12.04) in 
favor of treatment groups 1 and 2. There was a significant treatment x 
sex interaction (P < .05, F 2 ^i 99 = 3.32) in favor of boys on P501. The ad- 
justed means Indicated that students, in particular boys, who were given 
the Modified Wanted-Given Treatment were superior in achievement to those 
who were given The Productive Thinking Program and the control group in 
that a greater number of the students were evidently using a procedure 
which would lead to a correct solution. A separate analysis of covari- 
ance was run, with the control group omitted, to test the significance 
of the differences in means of Modified Wanted-Given Treatment over Treat- 
ment 1. The results were significant (P < .005, Fi^i46 = 6.23) in favor 
of Treatment 2, the Modified Wanted-Given Treatment. Again, there was a 
significant (P < .025, Fi^i 45 = 6.03) treatment x sex interaction in 
favor of boys. 

The results of an analysis of covariance on the follow-up scale 
Word Problems (F502) indicated a significant difference in favor of the 
treatment: groups (P < .001, F2^207 = 7.87) seven weeks after the main 
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TABLE II 

Adjusted Means on Word Problems (P50I) for Procedures 



Sex 


Productive Thinking 
Program 


Treatment 

Modified Wanted-Glven 
Program 


Control 




Boys 


2.07 


2.93 


1.60 




Girls 


2.28 


2.31 


1.75 





TABLE 12 

Adjusted Means on Word Problems (F502) for Procedures 







Treatment 




Sex 


Productive Thinking 
Program 


Modified Wanted-Glven 
Program 


Control 


Boys 


2.68 


2.96 


2.07 


Girls 


2.40 


2.66 


1.95 
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study had ended. The adjusted means for the posttest are shown in Table 
II and the adjusted means for the follow-up test are shown In Table 12. 
In every case Increase in adjusted mean scores from posttest to follow- 
up test was suggestive of some over-all effect, such as maturity or in- 
struction. There was no significant treatment x sex Interaction on F502 
for all three treatment groups. There were no significant differences 
for the two treatment groups on the follow-up F502 when an analysis of 
covariance was run with the control group deleted. There was a marginal 
sex effect on F502 (p <.10, Fi^i52 = 3.77) in favor of boys. Evidently 
the differences between the two treatment groups in terms of using cor- 
rect procedures decreased with the passage of time. It is Interesting 
to conq;)are the adjusted means for boys in both Treatment I and the con- 
trol. On the posttest the adjusted means for girls were higher in these 
two groups. In the follow-up test the adjusted means for the boys were 
higher. The adjusted means for girls Increased most for the Treatment 2 
group, but the scores for boys in this group did not Increase as much 
on.posttest means as did the scores for boys in the other groups. 

Taken as a whole, the results seem to indicate some degree of in- 
dependence of computational skill and solution strategy in solving pro- 
blems in arithmetic for fifth graders. 

; 

I 

i 

i 

I ■ . 

/ ■ 




CHAPTER IV 



SUMMARY AND DISCUSSION 

Rationale . The study was intended to examine which of tv/o ap- 
proaches, both successful in their own area, was the better method of 
problem solving in mathematics. Researchers, such as Olton, 

Crutchfield, and others have argued for the existence of a general pro- 
blem-solving ability and for the transferability of the general problem- 
solving skills developed through systematic instruction to specific dis- 
ciplines such as arithmetic. The results of several studies have con- 
firmed their position that students who receive instruction in The 
Productive Thinking Program apparently do become better problem solvers, 
while other studies have failed to find any significant differences . 
Failure to find significantly greater achievement levels in mathematics 
as transfer from training in The Productive Thinking Program , in studies 
to date, may have been due to the fact that the mathematical tasks used 
in some of these studies were too simple to demonstrate the full effects 
of the training. On the other hand, studies have shown that teaching stu 
dents a wanted-given method of analyzing word problems in arithmetic can 
produce significant differences in achievement in solving one-step pro- 
blems. This study atten^ted to provide a setting rich enough to test 
the hypotheses stated below. 

Purpose . The purpose of this study was to test the hypotheses 
that: (a) there will be no significant differences, in scores on posttest 
measures of problem-solving skills between students who receive training 
in general problem-solving techniques as presented in The Productive 



Thinking Program and students who are taught more specific skills for 
solving problems in a total mathematics context using a Modified Wanted- 
Given Program approach; (b) students in either of the treatment groups 
will not achieve significantly higher scores on measures of problem- 
solving skills than students in control classes who receive no special 
instruction in problem solving; and (c) there will be no significant 
difiierences between the achievement of boys and girls on tests of pro- 
blem-solving skills in the treatment groups. 

Subjects . The subjects in this study were 261 fifth-grade students 
from four schools in the San Jose area of Northern California. 

Treatments . The treatments consisted of; (a) The Productive 
Thinking Program Series One: General Problem Solving; (b) the author's 
Modified Wanted-Glven Program; and (c) the control group. Treatments 1 
and 2 were prepared in programmed booklet form. Each student completed 
one booklet each day for the 16 days of the instructional period. A 
pretest was administered before treatments began. A posttest was given 
the day following the end of the treatment period and a follow-up test 
was given seven weeks later. Students were assigned at random to treat- 
ments within classes in thi*ee of the schools. Students in the fourth 
school acted as a control. The results were analyzed using analysis of 
covariance. 

Results . The results are first summarized in terms of mean number 
of problems correct on posttests and follow-up tests and later in terms 
of the procedure used by students to solve word problems in arithmetic. 
On; the basis of the data from the present study, the results indicate 
t^t Hypothesis cannot be rejected. No significant differences due to 
treatment, at the .05 level or above, were found on any of the posttests 
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or follow-up tests. However, a difference at the .10 level favoring the 
Modified Wanted-Glven Program was found on the follow-up test Word Pro- 
blems (F502) Indicating some degree of superiority of this treatment over 
The Productive Thinking Program , which was given seven weeks after the 
posttest was given. 

Differences between means on parallel forms of the Word Problem 
test given as part of the posttest and seven weeks later as part of the 
follow-up test were not significant when subject to a t-test. 

Perhaps the best indicator of a possible reason for the low-level 
significance of the Modified Wanted-Glven Treatment on the Word Problem 
test is the difference in significance of covarlates for Word Problems 
between the posttest and follow-up test. The covarlate Working with 
Numbers did not have a significant relationship with the word problem 
scale on the posttest. The relationship was significant on the follow- 
up test. This finding tends to confirm an observation from the field; 
that la, the study was begun fairly early in the school year in an at- 
tempt to control any teacher-initiated Instruction on problem solving. 

A side effect was that the students involved in the study apparently 
had had Insufficient time to review their computational skills, so that 
their skills were actually below the level assumed by the writer when 
he wrote and tested the problems. While the results of the pilot tests 
gave no indication that the computational level was too difficult, 
close examination of the instructional booklets used in the Modified 
Wanted-Glven Treatment indicated that many of the children had difficulty 
multiplying by . two numbers, while others had difficulty with long divi- 
sion*. : ,-r, 
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Hypothesis 2 cannot be rejected. There were no significant differ- 
ences in achievement between treatment groups and the control group on 
any of the posttests or follow-up tests. While the adjusted means on 
the word problem test for the control group were somewhat lower than 
those for students in the Modified Wanted-Glven Program, the scores of 
the control students were higher in one case than for those students in 
The Productive Thinking Program . 

Of the six criterion tests, only one. Five Dots (X308), had a signi- 
ficant (P < .05) treatment x sex effect. In terms of "the statement of 
Hypothesis 3, this was nut considered sufficient evidence to warrant re- 
jection of the hypothesis . This was the only one of the six possible 
tests for \^lch significant sex effect was found, and since both Word 
Problem scales had low F values, Ifypothesls 3 should not be rejected. 

Two subsequent analyses, using covariance adjustment, of the proce- 
dures employed by students to solve problems in the Word Problem scales 
P501 revealed significant differences (P < .001) in favor of the treat- 
ment groups over the control, and a significant difference (P < .005) in 
favor of the Modified Wanted-Glven Program over The Productive Thinking 
Program . Students who were given the Modified Wanted-Glven Program treat 
ment tended to use correct procedures to solve the mathematical word pro- 
blems significantly more often than did their peers in either of the 
other two groups. This significant finding tends to confirm the valid- 
ity of the instructional approach used in the Modified Wanted-Giveh 
Program, at least in the short term. The significance level in favor 
of the treatment groups over controls on the follow-up test F502 was 
also .001. However, the difference in favor of the Modified Wanted- 
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Given Program over ^ Productive Thinking Program on the follow-up test 
was not significant at the .10 level, when comparing the two treatment 
groups. There was however, a treatment x sex Interaction (P < .10) in 
favor of boys in the Modified Wanted-Given Program. The effects of in- 
struction and possibly maturation during the interval before the follow- 
up test, may account for these findings, since all adjusted means were 
higher on the follow-up test than on the posttest. This may indicate a 
greater degree of independence between computational facility and problem- 
solving ability in terms of knowing how to go about solving a problem 
than was previously suspected. 

Limitations . The data analyzed and reported in the present study 
were collected from students in schools in a predominately white, 
middle-class socioeconomic area. All students in the study were fifth 
graders. Furthermore, the extremes in terms of gifted and handicapped, 
whether physical or educational, were not included in the data reported 
here. Any conclusions one might wish to draw must of necessity be made 
with the characteristics of the sample population clearly in mind. Per- 
haps the most important limitation was the finding that none of the 
pilot study groups reported the computational difficulty with the pro- 
blems presented in the Modified Wanted-Given Program treatment that 
students in the main experiment reported. Had the study been delayed 
until after Christmas, thus permitting teachers to complete a thorough 
review of the basic computational skills in each class, significant ef- 
fects in terms of the number of problems correct on each scale might 

have resulted. 

Conclusions and Implications . On the basis of the data presented 
and within their limitations, it would appear that teaching problem 
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solving in mathematics to fifth graders can best be done in a mathema- 
tical context using a wanted-given approach. One should not necessarily 
expect gains in the number of problems correctly solved, unless the 
teaching of problem-solving strategies is accompanied by instruction in 
the appropriate computational skills. The independence of these two 
factors seemed to be demonstrated when students in the experimental 
groups who were using a correct procedure to solve problems often failed 
to obtain significantly more mathematically correct answers, due to com- 
putational errors, than did the control group. 

Providing systematic instruction in problem solving was more effec- 
tive in this study in helping students to use correct procedures than 
not providing systematic Instruction. This would seem to say that those 
who contend that students learn to solve problems by simply doing lots 
of problems may in fact be taking the long way home. Students may learn 
to solve problems that way, but students in the present study who re- 
ceived either treatment used appropriate strategies much more often 
(P < .005) than did the control group. This does not mean however, that 
following any "systematic" program will produce better results than sim- 
ply having students solve lots of problems. 

Suggestions for research . Tho question of the significance of com- 
putational difficulty in problem solving in mathematics is an open issue. 
Students in this study were sometimes observed to block when faced with 
a difficult computational exercise, and as a result, were unable to pro- 
ceed at all toward the solution of the problem. 

The effect- of wording in problem statements is still' largely un- 
known. It is clear,' from the pilot; study wort however, that 

simply reordering the sentences; in a problem statement. can significantly 
affect the difficulty level of a problem. 
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The effect of a student's age and the approach used In this study 
remains open. Torrance, as mentioned earlier, suggested that after the 
sixth grade students appear to reach a plateau In problem-solving 
achievement when the treatment vehicle Is a series of progranned les- 
sons such as those used herein. His studies were all concerned with 
general problem-solving ability. It would be most interesting If one 
could determine age boundaries In the effectiveness of a programmed ap- 
proach to Instruction In problem solving In mathematics If Indeed such 
boundaries do exist. 

. A large number of other questions which relate to determining the 
relative difficulty of word problems remain to be answered. That is, 
what variables can one Identify that Influence the difficulty of word 
problems for students? Some of the variables that might be Investigated 
are the number of words used In the problem statement, the syntactic 
difficulty of the sentences themselves, the sequence of problems pre- 
sented, the number and type of operations required for solution, the 
memory load Imposed by the problem solution sequence, the discrepancy 
between the order In which the numbers are given In the problem state- 
ment and the order In which they must be combined In binary operations 
for the solution of the problem, and many others. Some of these vari- 
ables are currently being considered by researchers, and others will no 
doubt be considered In the near future. 

The suggestion of the semi-Independence of computational skill and 
procedural ability of students Is very Interesting. In most cases, in 
the author's experience, problem-solving tests are scored solely on the 
basis of the number of problems solved correctly, and no attempt Is 
made to examine the procedures students may have been using. It may be 




I 



86 






that students are often taught how to proceed to solve problems cor- 
rectly, but their new knowledge of how to solve a particular set of 
problems may be masked by the presence of computational errors. Com? 
putatlon and problem-solving ability may be highly correlated as many 
studies show, but the best measure of skill In problem solving will be 
one which brings to light the procedures used by students to solve pro- 
blems as well as to give the correct answers. Studies of the sort re- 
ported here are needed to confirm or counter the differences noted above. 
If the findings and conclusions are supported by other studies, then work 
should begin on the development of new measures of problem solvlng-- 
roeasures that evaluate both computation and procedures. 
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CHAPTER V 

REVIEW OF RESEARCH IN PROBLEM SOLVING 
ON A COMPUTER-BASED TELETYPE 



The first attempt at IMSSS to apply the linear regression model 
described in Suppes, Jerman, and Brian (1968) to the analysis of word 
problems in arithmetic was reported in Suppes, Loftus, and Jerman (1969)* 

A second study on word-problem variables using regression models was 
reported by Loftus (l970). What is desired from models of this sort is 
that they identify and define structural variables that are suspected to 
account for differences in the level of difficulty for a variety of types 
of word problems in terms of the structure of the problem itself. That 
is, one would like to attach weights to the several objectively defined, 
variables suspected of contributing to the relative difficulty of the 
problems, and then to use the estimates of the variables in predicting 
the relative difficulty of a large number of problems. 

A discussion of the regression model is not given here, since it 
has been described in detail in the three references cited above. Rather, 
the focus of this chapter is on a .review of the variables used in the 
first two of the previous studies, on their definitions, and on how 
well each accounted for the variance observed in djata from a new set of 
problems not solved at a teletype but in a regular classroom using paper 
and pencil. Some additional variables which were developed and used 
later are also discussed. 
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Xg Steps ; 



Length : 



The variables used in Suppes, Loftus, and Jerman ( 1969 , p. 7) were 
as follows. 

X^ Operations: the minimum number of different operations 

required to reach the correct solution; 
the minimum number of steps required to 
reach the correct solution; 
the number of words in the problem; 

Xj^ Sequential: assigned a value of 1 if the problem is 

of the same type (i.e., can be solved by 
the same operation(s) ) as the problem that 
preceded it, and 0 otherwise; 

Xc Verbal-cue: assigned a value of 1 if the problem con- 

tains a verbal cue to the operation(s) 
required to solve the problem, and 0 
otherwise; 

X^ Conversion: assigned a value of 1 if a conversion of 

units is required to solve the problem, 
and 0 otherwise. 

The 1969 study reported the results of the goodness of fit for 68 
word problems which '27 . gifted fifth-grade students solved as part of their 
daily CAl program. The regression equation for all 68 items was 

z . ' = -7 . 56 + , . 3TX*1 + . 18X^2 + ■ . 02X*^ + . 26X^^ + 1 . ^2X*g . 

The multiple R was .67 with standard error of 1.75> and an R of .^5* 

The value for the 68 items was 555*76, indicating a rather poor fit 
for the model . ' 



* 



indicates significance at the 65 level . 






)» ’ 7. '' . ’ » R ' lV. ’ P J ! i w p iwj r ^ !^ jR !Wt .wia-fVJ W n»wyi>j i > i ? wn ^wnc^tiaa l »n ^ \v w ra 



89 



Perhaps one of the variables in the above set which requires a 
description for clarification is X^, verbal cue. The following words 
were considered cues. 



Word 




Cue for 


and 




addition 


left 




subtraction 


each ■ 




multiplication 


average 






or 




division 



each 

The conversion variable was defined for the situation in which the 
student was to recall a fact from memory, such as 1 week = 7 days. This 
conversion, or substitution of units was a 0, 1 variable. 

Several people in the Institute became interested in formulating 
variables and testing their goodness of fit on the data from the Suppes, 
Loftus, and Jerman (1969) study. In particular. Dr. Barbara Searle 
formulated and tested several variables using the data set from the 
1969 study. The set of variables she tested consisted of a mixture of 
some of the above variables and some new variables which she formulated. 
The variables were defined as follows: 

Operations: the minimum number of operations required to 

reach a correct solution (values range 1-^4). 

Steps: the minimum number of binary operations, steps, 

needed to reach a solution (value range 1-7) • 



J!iength: 



the number of words in the 

7-51). 



problem (value range 
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Conversion: 



Verbal cue: 



Order: 



this factor is present if a conversion is required 
and the equivalent units are not given in the 
problem (a 0, 1 variable), 
the cue for each operation is as follows. 



Operation 
Addition: 
Subtraction: 
Mult ip 1 i cat i on : 
Division: 



Cue words 



added, altogether, gained 
how much less, lost, left 
each 
average 

If a cue word was present the value was 1, other- 
wise 0. 

If the steps to solution were in order as given 
in the problem statement, the value was 1, 
otherwise 0. 

If knowledge of a formula was required;, the value 
was i, otherwise 0. 

If the problem statement contained the word 
"average” the value was 1, otherwise 0. 

If the problem requires add it ion > the value is 1, 
otherwise 0. 

if the problem requires subtraction, the value is 
i, otherwise 0, . 

Multiplication ; If the problem' requires ' multiplication, the value 
, ' ' ■ " is i, otherwise 6. ' ^ . 

Division: If the problem requires division, the value is 1, 

otherwise 0. ' ' ' 



Formula : 



Average : 



Addition: 



Subtraction: 



m 
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Sequence: If the problem is in unusual order, the value 

is 1, otherwise 0. 

Three of the problems in the- original Set of 68 were , deleted due 

to their very high values. The following variables were tested on 

the data from the remaining 65 problems. 

Of the l6 variables in the expanded set, 12 were entered by the 

stepwise regression program, BMD 02 R. The value of the multiple R was 

.820. Three additional variables were formulated after studying the 

2 

weights of the variables, their contribution to the total R and their 
definitions. Two of these,. S'^ and S^, were sequential variables; the 
third was a memory variable. The definitions for the three additional 
variables follow.. . '• .. 

Memory (m) is defined as the sum of: 

C the number of conversions' + knowledge of formulas, 

D. the number of numerals in the problem statement, 

' 0 the number of different operations, 
is defined as the number of displacements of order of 
operations in successive problems. 

Examples: 



5 

5 



+ lf ) 

. kJ 






(5 + ^) X 2 ; 

(5 + h) -^2 ) 




5 + 5 V 

.'+,,1.6 J 




'(3 + i+) x'2,1 
(3 X 4) + 2 J 




Sg is defined as the number of displacements between order of 
operations required and that given in the problem statement 
itself. 



The results of the stepwise regression using all the variables which 

2 

contributed at least .01 to the variance in K on the data from the set 
of 65 original problems are preser^ed^ih Tqble 15. 
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TABLE 15 



Results of Stepwise Regression on Gy Word Problems 



A. Three additional variables 


alone 






Step 


Variable 


R 


Increase 

2 

in R 


Last reg. 
coefficient 


1 


19 memor 


.491 


.241 


0.106 


2 


17 si 


.?01 . 


.010 


0.073 


5 


18 s2 


.513 


.012 


0.087 


B. Over-all variables 

Step Variable 


R 


Increase 

2 

in R 


Last reg. 
coefficient 


1 


4 opers 


. 657 . 


.451 


.199 


2 


8 vblcu 


.697 


.055 


.262 


5 , 


15 divis* 


.729 


.046 


: .299 


4 


6 lentji 


.761 


.047 


.018 


5 


* 

10 formu 


.785 


.058 


.879 


6 


17 si 


ir\ 

0 

CO- 


' .052 


.150 


7 


7 convr 


.,825 


: , 052 ' 


.534 


8 


18 ;S2 


.855 ; 


.016 


.127 


9 


14 multi* 


.858 


•■005 


-.090 


10 


19 memor 


. 84 b 


.004 ■ 


-.048 


11 


12 add* 


.841 


.002 


. , . .065 


12 


16 seque 


. .842 

■I’ ' 


.001 


.053 



* 

Variables suggested by Dr. Sear le. 



V 'r *■’ I 
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■^1 Part A in Table I3 the last three variables are presented alone. 
The total R for the three was .51- Part B shows the results of the step- 
wise regression using all variables. Only those variables are considered 

2 

important which contributed to at least .01 increase in R . Although the 
value of the multiple-R term for all 12 variables was fairly high, .842, 
the number of vai-iables entered did not increase with the addition of the 
three new ones. Rather, the three new variables entered instead of others 

p 

and the resulting R increased from .820 to 842 (R = O.709). A total of 
19 variables had now been formulated and tested on the original data. In 
addition to those described above, the following were formulated and 
tested. 

Operations 2: The sum of the following. 

1. The number of different operations. ^ 

2. Add 4 if one of the operations is division. 

Add 2 if one of the operations is multiplication. 

Add 1 if onei of the operations is addition. 

Order 2: The sum of the following. 

1 - 

2. Verbal cue necessary to establish a new order. One 
point for each direct cue missing for each step. 

Recall: • The. sum of the following. • 

i. ■ 'Orii count for a formula to be recalled and one count 

for each step in the formula, e.g., A = 21 + 2w = 3 - 

'.o2. . , OneiOount for; each conversion to be recaHed and used. 

■ 3 * Ohb 'count for ' each fact recalled arid- hsed' from a 
previous problem. 
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Verbal cue 2: The set of cues was expanded. However, one count 

was given for each cue present in the problem. 
Addition: added, altogether, gained, total. 

Subtraction: how much less, lost, left 

how much larger, . . . than 

how much smaller . . . than 

/ how much greater . . . than 

hew much further . . . than 

Multiplication: each, times 

Division: average 

Distractors: This variable was defined as one count for each 

verbal cue which was not a cue for an operation, 
but a distractor. For example, if the word 
"average” was used, but multiplication rather than 
division was the req.uired operation. 

A complete list of variables, by number, follows. 



Variable 


Name 


Variable 


Name 


1. 


P( correct-observed) 


12. 


Multiplication 


2. 


Operations 


13. 


Division 


■ 


Steps, 


. l4. 


, Seq^uence 


4. 


Length 


15. 




5. 


Conversions 


l6. 


.. 

^2 


6. 


Verbal Cue 


17. 


Memory 




Order , 


l8. , 


Operations-2 


8. 


Formula 


19. 


Order-2 




Average 


20. • 


■ Recall 


■'10. ^ 


Addition 


-■k':.' 21.' 


jt. Verbal Cue-2 




, V Subtiraction 


OJ 

CM 


, Distractor Cue 



: .r.'j.l-tiV'.n-Gj’ ’i-uo j,vv 






95 



2 

It appears from observing the values of R and the increase in R 
with each step that there is relatively little gain after the tenth 

step in either Table I3 or Table 14-. In fact if one were to adopt the 

rule to consider as important only those variables whose contribution 

2 

to the increase in R was .01 or greater, then the first eight variables 
in Table I3 and the first nine variables in Table l4 would comprise the 
set of variables of interest in each case. 

It is most interesting to examine the order of entry of the vari- 
ables in each case, and at this stage, it is impossible to explain the 
differences. About all we can say to the reader is "behold." The 
following list ranks the variables under discussion for ease of com- 
parison. 



From Table I3 



From Table l4 



Step 

1 



Operations- 



R 

-.657- 



Step 

— 




R 

Operations .657 
Conversions .J02 
Length .7^0 
Order 2 .780 
Division .805 

Sg .821 

Order .829 
Memory . 835 
Distractor Cues .84l 



At this point, note that the variables which appear to be the 
most robust are length, division, Sg the internal sequence variable, 
and conversions. Memory and distractor cues may or may not play 
important roles ill subsequent analyses for students working at teletype 



terminals. 
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table ik 

Summary Table for the Stepwise Regression on 65 Items 



Step 

Num. 


Name 


Variable 
Ent. Rem. 


Multiple 

R 


Increase 
in R^ 


F Value 
For Del 


Last Reg. 
Coefficients 


1 


Oper 


2 


0.6566 


0.4511 


. 0.4511 


47.752 


0.1824 


2 


Convr 


5 


0.7021 


0.4929 


0.0618 


7.559 


0.5517 


5 


Lenth 


k 


0.7^05 


0.5485 


0.0554 


7.488 


• 0.0274 


k 


Order 2 


19 


0 . 780 J+ 


0.6090 


0.0607 


9.521 


0.1427 


5 


Divis 


* 

15 


0.8050 


0.6480 


0.0590 


6.516 


0.5841 


6 


So 


16 


0.8215 


0.6745 


0.0265 


4.749 


0.1791 


7 


2 

Order 


7 


0.8288 


0*. 6869 


0.0124 


2.252 


- 0.1401 


8 


Memory 


17 


O.Q3k9 


0.6971 


0.0102 


1.872 


-0.0859 


9 


List 


22 


0 . 81+09 


0.7071 


0.0101 


1.984 


-0.5052 


10 


Re cal 


20 


0 . 841 + 1 + 


0.7150 


0.0059 


1.105 


0.1455 


11 


Verb 1 C 


6 


0 . 8457 . 


0.7152 


0.0022 


0 . 4 l 4 


-0.1055 


12 


Verbl 2 


21 


0.8477 


0:7186 


0.0054 


Q '.629 


0.1550 


15 


Si 


1 ? 


0.8498 


0.7222 


0.0056 


o, 64 l 


0.0482 


ih 


1 

Aver 


* 

9 


0.8504 


0.7252 


0.0010 


0.202 . 


-0.1551 


15 


Oper 2 


18 


0.8515 


0.7247 


0.0015 


0.261 


-0.0528 


16 


Sub 


* 

11 


0.8520 


. 0-7259 


0.0012 


0.197 


-0.0607 


17 


Seq. 


Ik - 


0.8527 


0.727.1 


0.0012 


. 0.210 


0.0865 


18 


Steps 


5 ■ 


0.8529 


' 0.7274 


0.0005 


0.065 


0.0506 



Variables suggested by Dr.-Searle. 









rOTUfTW 



yrfinrtr'ntr. 






:»7y* R yT WAi »?; 



CHAPTER VI 



APPLICATION OF STRUCTURAL VARIABLES TO WORD PROBLEMS 



SOLVED IN . A PAPER-AND-PENCIL CONTEXT 



The research reported in Chapters I-IV was conducted entirely off- 
line, in a paper-and-pencil mode. In the course of the treatment, 
students worked through several sets of word problems that were evaluated 
as tests of their achievement, but were not labeled as tests. .Other sets 
of word problems were labeled as tests and were included as part of the 
posttest and follow-up test instruments. The following is a report of 
an analysis, using the variables defined in Chapter Y, of one set of 
word problems that tested the generalizability of the model to off-line 
situations . 

Twenty-nine problems were selected for analysis. Of these problems, 
19 were solved by students (N = 20) using paper and pencil. The I 9 prob- 
lems were part of the instructional treatment provided by the Modified 



Wanted-Given Program, and the remaining 10 problems came from the two 



the earlier analyses were tested on the word problems solved off line 
to see if their order of entry in the stepwise regression was at all 
similar. The variables selected for testing were the following. 



test scales P501 and F502 (N = 161 ).. -Eleven of the variables used in 



2. Operations 2 



3 . Order 2 



8 . Length 

9 . Verbal Cue (as defined by 




Dr. Searle) 



ll-. Recall 



10. Conversion 
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5. S. 



11 . Formula 

12. Division (as defined by Dr. Searle) 



6 . Memory 

7. Sg 

The results of the stepwise regression using these variables are 
summarized in Table 15. As can be seen, Length, the number of words in 
the statement of the problem, entered first followed by Memory, Sg, 
and Verbal Cue. The total R after nine steps was .77j R = *595* This 
was somewhat surprising and gratifying. The variables accounting for 
most of the variance on line were also effective, though at a lower level, 
for accounting for most of the variance off line where the students did 
all the required computation by hand. The observed and predicted 
probability correct for each item is shown in Table l6. 

The conclusions must be recognized as artificial, however. While 
the students did in fact solve all 29 problems, they solved only the first 
19 at one sitting. The last 10 problems were solved 5 at a time, as post- 
test and follow-up tests, seven weeks apart. The sequential variable is 
not correct, therefore:, for problems 19 and 25. Otherwise the variable 
values are the same as if students had taken the entire set of problems 
at one time. However, the results cannot be interpreted as being constant 
for students who would solve all 29 items at one sitting. Only the values 
of the variables are the same. 
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TABLE 15 

Summary Table for 29 Word Problems Administered in 
Paperrandrpencil Format, 11 Original Variables 



Step 

Num. 


Name 


Variable 
Ent. Rem. 


Multiple 
R R^ 


Increase 

2 

in BT 


F . Value 
For Del 


Last Reg. 
Coefficients 


1 


Lenth 


8 


0.5487 


0.3011 


0.3011 


11.633 


0.0207 


2 


Memor 


6 


0.5938 


0.3526 


0.0515 ' 


2.069 


0.1588 


3 


Sg 


7 


0.6672 


0.4455 


0.0926 


4.167 


-0.5048 


4 


\ 


5 


0.6977 


0.4868 


0.04l6 


1.9‘^9 


-0.1579 


5 


Vblcu 


9 


0.7399 


0.5475 


0.0607 


3.082 


0.5784 


6 


Oper 2 


2 


0.7549 


0.5699 


0.0224 


1.148 


0.0962 


7 


Recal 


4 


0.7661 


0.^5869 


0.0170 


0.863 


-0.2790 


8 


Convr 


10 


0.7708 


0.5941 


0,0072 


0.356 


-0.0797 


9 


Divis 


12 


0.7713 


0,59^9 


0.0008 


0.038 


-0.0857 
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TABLE l6 

Observed and Predicted Probability Correct for 29 



Word Problems, 11 Original Variables 



Case Num. 


Observed 


Predicted 


Residual 


1 


0.2660 


0.2116 


0.0544 


2 


O.2U1O 


0.4491 


-0.2081 


5 


0.5100 


0.5457 


-0.0337 


k 


0.6200 


0.5855 


0.2365 


5 


0.2500 


0.4580 


-0.2280 


. 6 


0.5720 


0.4066 


-0.0346 


7 


0.0100 


0.1297 


0.3703 


8 


0.7770 


0.6099 


0.1671 


9 


O.I7UO 


0.2201 


-0.0461 


10 


0.5600 


0.5600 


0.0000 


11 


0.5580 


0.5542 


0.1838 


12 


0.1150 


0.1681 


-0.0531 


15 


0.0870 


0.0870 


0.0000 


lU 


0.1000 


0.1692 


-0.0692 


15 


0.0710 


0.1180 


-0.0470 


l6 


0.2940 


0.2013 


0.0927 


17 


0.1660 


0.2959 


-0.1279 


18 


0.0100 


0.5333 


-0.0333 


19 


0.0590 


0.0989 


-0.0399 


20 


0.4580 


0.3783 


0.0597 


21 


0.2440 


0.5652 


-0.1212 


22 


0.4540 


0.5325 


0.1215 


23 


0.5550 


0.3967 


-0.0417 


2 k 


0.1050 


o.i44o 


-0.0390 


25 


0.3410 


0.3783 


-0.0373 


26 


0.5770 


0.5652 


0.0118 


27 


0.5170 


0.5744 


0.1426 


28 


0.3870 


0.5967 


-0.0097 


29 


0.1610 


0.1228 


0.0382 
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In an attempt to improve the fit, two new variables were defined. 
The first. Verbal Cue-1 (No. 13 was a redefinition of Dr. Searle's 
variable. It was essentially the definition used in Verbal Cue-2 except 
that it was a 0-1 variable rather than a frequency variable as is Verbal 
Cue 2. The second new variable (No. l4) was a combination of Verbal Cue 
1 and indirect cues, such as "in all," for addition, "short of ...," for 
subtraction and "per ...," for multiplication. The results of the regres- 
sion analysis on the 29 problems using l4 variables are shown in the 
summary Table 17. The increase in the value of R due to the addition 
of the two additional variables was small, almost .03 (.771 to .799)* 

The observed and predicted scores for each problem are shown in Table l8. 

TABLE 17 

Summary Table for 29 Word Problems Alministered in 



Paper-and-pencil Form, 13 Original Variables 



Step 

Num. 


Variable 
Ent. Rem. 


Multiple 
R R^ 


Increase 

2 

in 


F Value 
For Del. 


Last Reg. 
Coefficients 


1 


8 


0.5487 


0.3011 


0.3011 


U.633 


0.0270 


2 


6 


0.5938 


0.3526 


0.0515 


2.069 


0.2115 


3 


7 


0.6672 


0.4452 


0.0926 


4.167 


-0.2184 


4 


5 


0.6977 


0.4868 


0.04 16 


1.949 


-0.1845 


5 


9 


0.7399 


0.5475 


0.0607 


3.082 


0.5859 


6 


14 


0.7688 


0.5911 


0.0436 


2.350 


-0.0761 


7 


2 


0.7872 


0.6197 


0.0286 


1.577 


0.0329 


8 


10 


0.7964 


0.6343 


0.0146 


0.797 


-0.1165 


9 


13 


0.7974 


0.6359 


0.0016 


0.080 


-0.1453 


10 


12 


0.7986 


0.6378 


0.0019 


0.100 


0.1586 


11 


4 


0.7991 


0.6386 


0.0008 


0.034 


-0.1257 


12 


3 


0.7993 


0.6389 


0.0003 


0.013 


-0.0085 
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Two variables described earlier. Verbal Cue-2 (No. 15) and a 
distractor variable (No. l6), were added to the regression. The results 
of the regression using all 15 variables are shown in Tables 19 and 20. 
The value of R increased from .799 with 13 variables to .834 with 15 
variables. 

The practicality of taking into account more than four or five 
variables when writing word problems is impossible, even if they were 
able to account for a larger portion of the variance than that indicated 
in Table 19* Clearly, the fit of the variables selected and tried thus 
for was less than satisfactory. The fact that the off-line students 
performed all computations by hand led to the definition of four new 
computational variables. These followed the work reported in Suppes 
and Morningstar (in press. Chapter III). The variables were: 

17. EXMC. A 0-1 variable. A count of 1 was assigned for 
each multiplication exercise required in the solution 
of the problem. Zero was assigned if multiplication 
was not required. 

18. N0MC2. A count of 1 was assigned each time a regrouping 
occurred In' each multiplication exercise in the problem. 

For example: 



38 

>L1 

190 NOMC = 2 

38 
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TABLE 19 



Summary Table for 29 Word Problems 



in Paper -and-pencil Formal, 15 Original Variables 



Step Variable 

Num. Ent. Rem. 


Multiple 
R R^ 


Increase 

2 

in R 


F Value 
For Del. 


Last Reg. 
Coefficients 


1 


.5 


0.5487 


0.5011 


0.5011 


11.655 


0.0408 


2 


6 


0.5958 


0.5526 


0.0515 


2.069 


0.1712 


5 


7 


0.6672 


0.4452 


0.0926 


4.167 


- 0.5204 


k 


15 


0.7180 


0.5155 


0.0704 


5.488 


-0.2857 


5 


5 


0.7465 


0.5575 


0.0417 


2.167 


- 0.2402 


6 


12 


0.7667 


0.5878 


0.0506 


1.629 


- 0.2402 


7 


10 


0.7798 


0.6081 


0.0205 


1.089 


-0.1611 


8 


9 


0.7912 


0.6260 


0.0179 


0.954 


0.7250 


9 Vbl 5 


5 


0.8077 


0.6524 


0.0264 


1.445 


0.0946 


10 Vbl 4 


4 


0.8169 


0.6675 


0.0150 


0.807 


0.4544 


11 


l 4 


0.8500 


0.6889 


0.0216 


1.180 


-0.1815 


12 


15 


0.8552 


0.6942 


0.0055 


0.285 


0.1214 


15 


12 


0.8552 


0.6942 


0.0000 


0.004 




Ik 


16 


0.8541 


0.6957 


0.0015 


0.079 


-0.0520 


15 Vbl 2 


2 


0.8545 


0.6961 


0.0005 


0.014 


0.0119 



00 ^ 
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TABLE 20 





Observed and Predicted 


(Correct) for 15 


Variables 


Case Kum. 


Observed 


Predicted 


Residual 


1 


0.2660 


0.2421 


0.0259 


2 


0.2410 


0.5279 


-0.2869 


5 


0.5100 


0.2968 


0.0152 


4 


0.6200 


0.4579 


0.1821 


5 


0.2500 


0.5557 


-0.1057 


6 


0.5720 


0.2751 


0.0969 


7 


0.5000 


0.1966 


0.5054 


8 


0.7770 


0.6401 


0.1569 


9 


0.1740 


- 0.2482 


-0.0742 


10 


0.5600 


0.5455 


0.0165 


11 


0.5580 


0.4196 


0.1184 


12 


0.1150 


0.1912 


-0.0762 


15 


0.0870 


0.0870 


0.0000 


14 


0.1000 


0.1062 


-0.0062 


15 


0.0710 


0.1067 


-0.0557 


16 


0.2940 


0.2847 


0.0095 


17 


0.1660 


0.4012 


-0.2552 


18 


0.5000 


0.5462 


-0.0462 


19 


0.0590 

• 


0.0676 


-0.0086 


20 


0.4580 


0.5859 


0.0541 


21 


0.2440 


0.2646 


-0.0206 


22 


0.4540 


0.5912 


0.0628 


25 


0.5550 


0.5854 


-0.0504 


24 


0.1050 


0.1450 


-0.0400 


25 


0.5410 


0.5859 


-0.0429 


26 


0.5770 


0.2646 


0.1124 


27 


0.5170 


0.4822 


0.0548 


28 


0.5870 


0.5854 


0.0016 


29 


0.1610 


0.1048 


0.0562 
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19. COLC. For this variable a count of 1 was given for each column 
and a count of 1 was given for each regrouping in the largest 
addition and subtraction exercise in the problem, If no 
addition or subtraction was required, a count of 0 was given. 

20. QUOT. A count of 1 was given for each digit in the quotient 
if division was required and 0 otherwise. 

The set of 19 variables was coded for use on the set of 29 problems 
which had been rearranged from highest to lowest in order of probability 
correct. One additional problem (P(correct) = .50) was added to bring 
the total number of problems up to 30. The set of 30 problems was being 
prepared for administration to several groups of new students for repli- 
cation of the earlier analyses. 

A regression was run bn the 30 problems using all 19 variables. 

The summary of this analysis is presented in Table 21. The value of R 
after the first five steps was 0.9315, R = 0. 86769. This is a surprisingly 
good fit for Just five veiriables. Perhaps even more suprising was the 
strength of the computational variables indicated by their point of entry 
into the regression program. Of the first five variables which entered 
the regression, three were computational variables — NQMC a multiplication 
variable, QUOT a division variable, and COLC an addition and subtraction 
veuriable. The variable LENTH which accounted for the number of words in 
the problem statement entered first and the distractor variable DIST 
entered on the fourth step of the regression. The cognitive variables 
such as memory and order did not enter as soon or in the same order for 
students solving problems at a CAI terminal. The observed and predicted 
probability correct for each problem are presented in Table 22. 
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TABLE 21 



Summary of Regression Analysis of 50 Problems 
Using - 19 Variables 



step 

Num. 


Variable 
Ent. Rem. 


Multiple 

R 


Increase 
in R^ 


F Value 
For Del. 


Last Reg. 
Coefficients 


1 


8 


0.6684 


. 0.4468 


0.4468 


22.609 


0.0179 


2 


18 


0.8595 


0.7048 


0.2580 


23-586 


0.2419 


5 


20 


0.9129 


0.8334 


0.1286 


20.067 


0.2323 


4 


16 


0.9254 


0.8564 


0.0230 


4.013 


0.1333 


5 


19 


0.9315 


0.8677 


0.0113 


2.055 


0.0684 


6 


5 


0.9350 


0.8742 


0.0065 


1.189 


0.0421 


7 


7 


0.9400 


0.8836 


0.0094 


1.761 


-0.1679 


8 


VBL4 4 


0.9428 


0.8889 


0.0053 


1.012 


-0.3726 


9 


17 


0.9462 


0.8953 


0.0064 


1.206 


0.0896 


10 


• 6 


0.9470 


0.8968 


0.0015 


0.303 


-0.0902 


11 


VBL2 2 


0.9477 


0.8981 


0.0013 


0.220 


0.0600 


12 


VBL5 3 


0.9487 


0.9000 


0.0019 


0.329 


0.0580 . 


15 


15 


0.9499 


0.9023 


0.0023 


0.362 


-0.1434 


14 


14 


0.9532 


0.9086 


0.0063 


1.052 


0.0880 


15 


10 


0.9541 


0.9103 


0.0017 


0.260 


0.0565 


16 


9 


0.9542 


0.9105 


0.0002 


0.038 


-0.0711 


17 


13 


0.9543 


0.9107 


0.0002 


0.010 


0.0298 
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TABLE 22 



Observed and Predicted Probability Correct 
for Each Problem 



Case Num. 


Observed 


Predicted 


Residual 


1 


0.7770 


0.6645 


0.1125 


2 


0.6200 


0.5727 


0.0475 


5 


0.5600 


0.5600 


0.0000 


4 


0.5580 


0.3721 


0.1659 


5 


0.5170 


0.4585, 


0.0585 


6 


0.5000 


0.5510 . 


-0.0510 


7 


0.4540 


0.4457 


0.0083 


8 


0.4580 


0.5774 


-0.139^ 


9 


0.3870 


0.3254 


0.0616 


10 


0.3770 


0.1840 


0.1930 


11 


0.3720 


0.6047 


-0.2327 


12 


0.3550 


0.3545 


0.0205 


15 


0.3410 


0.4647 


-0.1237 


14 


0.3100 


0.1917 


0.1183 


15 


0.2940 


0.2955 


-0.0013 


16 


0.2660 


0.2168 


0.0492 


17 


0.2440 


0.2957 


-0.0517 


18 


0.2410 


0.2779 


-0.0389 


19 


0.2300 


0.2740 


-0.0440 


20 


0.1740 


0.1845 


-0.0105 


21 


0.1660 


0.1517 


0.0343 


22 


0.1610 


0.1497 


0.0112 


25 


0.1150 


0.1427 


-0.0277 


24 


0.1050 


0.1555 


-0.0463 


25 


0.1000 


0.1082 


-0.0082 


26 


0.0870 


0.0870 


0.0000 


27 


0.0710 


0.0813 


-0.0103 


28 


0.0590 


0.0584 


0.0006 


29 


0.0100 


0.0153 


-0.0053 


50 


0.0100 P 


0.0098 


0.0002 
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Only the first five variables contributed to an increase in R of 
one per cent or more. The regression equation after the fifth step was 

= -.75 + .02X2 *22X4 + .05X5 + -23Xg . 

Table 23 presents the regression coefficients, standard errors of 
regression coefficients, and computed T- values for each of the five 
variables. Only one of the variables, COLC, failed to reach significance 
at the .05 level. 



TABLE 23 



Regression Coefficients, Standard Errors of Regression 
Coefficients, and Computed T- values 





Variable 


Regression 

Coefficient 


Standard 

Error 


Computed 

T-value 


*2 


length 


.01955 


.00626 


3.10703’^ 


X, 


distractor 


.18769 


.08924 


2.1003* 


3 

X4 


multiplication 


.22076 


.02705 


WWW 

^ n II II 

8.1537 


*6 


addition- subtraction 


.02819 


. 01966 


1.4847 


division 


.22983 


.05979 


*** 

3.8429 



P < .05 



## 



P < .01 






p < .001 



The correlation matrix for the veuriables is shown in Table 24. 
Variable one is the observed percentage correct and variable seven is 
the transgenerated dependent variable. The mean and standard deviations 
for each variable are given in Table 25. The summary for the five-step 
regression analysis is given in Table 26. 
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TABLE 2 k 



Correlation Matrix for Each Variable 



Variable 

Number 


1 2 


3 


4 


5 


6 


7 


1 


1.000 - 0.637 


- 0.533 


- 0.413 


0.001 


-0.454 


-0.926 


2 


1.000 


0.610 


- 0.007 


- 0.205 


0. 606 


0.663 


3 




1.000 


- 0.145 


- 0.147 


0.605 


0.543 


4 






1.000 


-0.090 


-0.226 


0.503 


5 








1.000 


-0.118 


-0.092 












1.000 


0.566 














1.000 



TABLE 25 

Means and Standard Deviations for Each Variable 



Variable 


Mean 


Standard 

Deviation 


P(C) 1 


29.59667 


19.55739 


Lenth 2 


35.73333 


9.63447 


Distr 3 


0.33333 


0.66089 


Nome 4 


1.46667 


1.67607 


Cole 5 


2.13333 


2.25501 


Quot 6 


0.43333 


1.00630 


7 


0.51014 


0.58148 
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TABLE 26 



Summary of Five-step Regression Analysis 


Multiple R 


0.9315 






Std. Error 


of Est. 0.2325 






Analysis of Variance: 








DF Sum of Squares 


Mean Square 


F- Ratio 


Regression 


5 8.508 


1.702 


31.485 


Residual 


24 1.297 


0.054 




Variables in Equation: (Constant = • 


-0.73442) 




Variable 


Coefficient 


Std. Error 


F to Remove 


Lenth 2 


0.01955 


O.OC626 


9.7476 (2) 


Distr 5 


0.18769 


0.08924 


4.4233 (2) 


Nome 4 


0.22076 


0.02705 


66.6153 (2) 


Cole 5 


0.02819 


0.01966 


2.0545 (2) 


Quot 6 


0.22983 


0.05979 


14.7775 (2) 
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In summary, the linear regression model described in this paper has 
given a surprisingly good account of the difficulty level of a somewhat 
artificially arreuaged set of verbal problems for fifth-grade students. 
Five variables were found to account for almost 87 per cent of the 
variance in the observed probability correct. The variable which 
accounted for most of the variance was ROMC (32 per cent) a multiplication 
variable, followed by QLIOT (23 per cent) a division variable, then LHTEH 
(21 per cent) the number of words in the problem statement, DIBTR (ll per 
cent) the verbal distractor variable, and finally COI/! (1 per cent) the 
addition-subtraction vau'iable. 

Before one tries to apply these findings to a real-life situation, 
however, a good deaO. of replication needs to be done to either confirm or 
deny the findings reported here. For example, which variables enter first 
for students at different grade levels or for students with different 
back^ounds or skill levels. Further, variables for students solving 
problems on a CAI system that eliminates the need for computational 
facility should be compared with the variables that were found to account 
for most of the variance in this study, i.e., for students required to do 
their own computations. Perhaps we can then get a clear picture of the 
distinction between levels of cognitive skills. It may be that stepwise 
regression techniques will provide a more suitable vehicle for determining 
whether aptitude- interaction factors do exist. 

One indication of the differential effect among the several 
variables for on-line and off-line problem solving is indicated by the 
data shown in Table 27. 'Hie set oT variables found so effective in 
predicting the probability correct for the set of 30 word problems was 
tested on the data frem the Suppes, Loftus, and Jerman (1969) study. 
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As can be seen frou Table 27 j the overall fit is not as good and the 
order of entry of the variables is quite different from that shown in 
Table 21. 

The first five variables from each analysis are shown below. 

The multiple R after five steps is 

Step From Table 21 From Table 27 



1 


Lenth 


Oper 2 


2 


Nome 


Lenth 




Quo 


Ordr 2 


h 


Distr 


Recal 


5 


Cole 2 


^2 


R 


.932 





also given. The data from Table 21 were based on paper-a .d-pencil work 

and the data fron Table 27 on CAI work. It is clear from a study of 

the tables that the same variables do not enter in the same order or 

2 

account for the same amount of increase in R in each case. The 
variable Lenth does enter early in each case, however, and seems to be 
a consistent early entrant. 

It is unfortunate that the intact set of problems analyzed in 
Table 21 was never run on the CAI system so that the variables entered 
both on line and off line could be analyzed. Many of the thirty problems 
in the set were included in the problem-solving strand, but upon entry 
the problem set was split and items were scattered throughout the strand 
according to another set of variables all without the author's knowledge. 

This made comparison for purposes of the present study practically impossible. 
Nevertheless, it does appear that the variables affecting item difficulty 
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TABLE 27 

Computational Variables Run on Suppes, Loftus 
and Jeroan (1969) Data 



Step 

Hum. 


Name 


Variable 
Eat. Rem. 


Multiple ^ 

R r< 


Increase 
in R^ 


P Value 
: /r Del 


Last Reg. 
Coefficients 


1 


Oper 2 


8 


0.5722 


0.3274 


0.3274 


30.674 


0.0427 


2 


Lenth 


2 


0.6539 


0.4276 


0.1002 


10.844 


0.0266 




Ordr 2 


9 


0.7070 


0.4999 


0.0723 


8.809 


0.1543 


U 


Recall 


10 


0.7367 


0.5427 


0.0429 


5.641 


0.2526 


5 


Sg 


6 


0.7466 


0.5574 


0.0147 


1.949 


0.1276 


6 


Nome 


12 


0.7607 


0.5787 


0.0213 


2.923 


-0.5043 


7 


Memory 


7 


0.7725 


0.5968 


0.0181 


2.555 


-0.0440 


8 


Cole 


15 


0.7768 


0.6034 


0.0067 


0.938 


0.1212 


9 


Divis 


3 


0.7782 


0.6056 


0.0029 


0.310 


0.1165 


10 


Vblc 2 


11 


0.7796 


0.6078 


0.0029 


0.304 


-0.0576 


11 


(^0 


16 


0.7803 


0.6089 


O.OOU 


0.133 


0.0111 


12 


Sj. 




0.7806 


0.6093 


0.0005 


0.080 


-0.0268 


13 


Seq 


4 


0.7810 


0.6100 


0.0006 


0.071 


0.0458 
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115 



are different when comparing problems that do not require computation 
with problems that do require computation. 

It is too early to attempt an exploration of the observed 
differences in the entry iHsints of the variables used. More work needs 
to be done. However, preliminary evidence suggests that one must not 
Jump to conclusions about which variables are the most important con- 
tributers to the difficulty of verbal problems in arithmetic without 
first considering the tools at the disposal of the student who is to 
solve the problem. 
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